Probability & Random Processes
These notes are for the version of the class which is taught by professor Yates in the spring of 2016.
Outline
 Chapter 1  Set Theory
 Chapter 2  Tree Diagrams and Sequential Experiments
 Chapter 3  Discrete Random Variables
 Chapter 4  Continuous Random Variables
 Chapter 5  Multiviews and Pairs of Random Variables
 Chapter 6  Probability Models of Derived Random Variables
 Chapter 7  Conditional Probability Models
 Chapter 9  Sums of Random Variables
 Chapter 10  The Sample Mean
Chapter 1  January 21st 2016
We’re going to start first with Set Theory
Set Theory
A set is a collection of things. We usually will use capital letters to denote sets
parts of a set are called elements. When we use mathematical notation to denote an element as being part of a set we use the following symbol \in
Example:  x belongs to A
Set Unions
The union of sets and is the set of all elements that are either in A or in B or in both The union of A and B
Set Intersections
The intersection of sets is set of elements where each elements present in the set of A is also contained in B. We use the symbol to show the intersection of two sets.
Note!: Typically the is not used in problems and the following statements are equivalent:
Mutually Exclusive set what we call two sets with no common elements
We call the universal set the set which contains everything. That is if we imagine all of our elements being made up of sets then we get
A partition is a set which is mutually exclusive and collectively exhaustive part of S.
Theorem 1.1
De Morgan’s law relates all three basic operations
This can be proved by using the following logical steps
 Suppose x belongs the complement of the union of A and B
 Step 1 implies that x does not belong to the union of sets A and B
 Step 2 implies that x does not belong to set A nor set B.
 Step 3 implies that x must then belong to the complement of set A and the complement of set B.
 Step 4 Then must imply that x is contained within the intersection of sets and .
Applying Set Theory to Probability
An experiment consists of a procedure and observations. The is uncertainty in what will be observed; otherwise, performing the experiment would be unnecessary
an outcome of an experiment is any possible observation of that experiment
The sample space of an experiment is the finestgrain, mutually exclusive, collectively exhaustive set of all possible outcomes.
 Sample space = universal set
an event is a set of outcomes of an experiment
Probability Axioms
A probability measure P[.] id s function that maps events in the sample space to real numbers.
 Axiom 1  For any event A,
 Axiom 2 
 Axiom 3 For any countable collection of mutually exclusive events:
Theorem 1.3
if and for .
In other words
Theorem 1.4
The probability measure satisfies the following:
 For any A and B (not necessarily mutually exclusive)
 .
Theorem 1.5
The probability of an event is the sum of the probabilities of the outcomes contained in the event:
Theorem 1.6
For an experiment with sample space in which each outcome is equally likely:
Section 1.4  Conditional Probability
Definition, Conditional Probability: The probability of the event A given the occurrence of event B is:
The vertical bar, “” stands for the word “given”.
What this equation does is “scale up” the probability of all events inside the event B. This effectively makes B the “new” sample space because all events that occur after B is given must be contained within B
Theorem 1.7
A conditional probability measure has the following properties that correspond to the axioms of probability.
 If with for then
Partitions and the Law of Total Probability
Theorem 1.8
For a partition and any event A in the sample space, let for , the events are mutually exclusive and:
Theorem 1.9
For any event A, and partition
Theorem 1.10
For a partition with for all i,
Theorem 1.11: Bayes’ Theorem
What this is saying is that the probability of an event B, given the event A, will be exactly equal to the probability of A divided by the probability of B, multiplied by the probability of A given B
Section 1.6 Independence
Definition: Independence  Events A and B are independent if and only if:
A comment on event independence and mutually exclusivity. The two are not synonyms. In some contexts they can have similar meanings. But in probability this is not the case.
Mutually exclusive events A and B have no outcomes in common and therefore . In most situations independent events are not mutually exclusive.
Exceptions occur only when . When we have to calculate probabilities, knowledge that events A and B are mutually exclusive is very helpful. Axiom 3 enables us to add their probabilities to obtain the probability of the union. Knowledge that events C and D are independent is also very useful. Definition 1.6 enables us to multiply their probabilities to obtain the probability of the intersection.
Definition 1.7  Three Independent Events
Three events are mutually independent if and only if:
 are independent
 are independent
 are independent
and finally as long as:
Definition 1.8 More than two Independent Events
This definition is the same as above. All collections of events must be mutually exclusive of one another,
The probability of intersections of these events must be equal to the product of their individual probabilities.
Chapter 2  Tree Diagrams
Below is an example problem which will introduce us to tree diagrams
From this we can see that tree diagrams help us to calculate probabilities of sequential events.
Tree diagrams display the outcomes of the subexperiments in a sequential experiment. The labels of the branches are probabilities and conditional probabilities. The probability of an outcome of the entire experiment is the product of the probabilities of branches from the root of the tree to a leaf.
Many experiments will consist of what we call subexperiments. The procedure followed for each subexperiment may depend on the results of the previous subexperiments.
In doing this we will assemble outcomes into different partitions, starting at the root of the tree. From there we branch (and label according probabilities) based on the probabilities of each outcome.
Section 2.2  Counting Methods
In all application of probability theory, it should be important to understand the sample space of an experiment.
The methods in this section will help determine the number of outcomes in the sample space of an experiment.
Theorem 2.1
An experiment consists of two subexperiments. If one subexperiment has outcomes and the other subexperiment has outcomes, then the experiment has a total of outcomes.
Theorem 2.2
The number of permutations of distinguishable objects is:
Something that might help to remember permutations is that a permutation depends on the order in which the elements are in whereas a combination does not depend on the order.
For example, think if we had 100 students. Imagine that we want to put any 50 of those students into a line. This means that the number of different permutations (arrangements) of that line is equal to
Theorem 2.3
The number of ways to choose k objects out of n distinguishable objects is:
Sometimes this is called a combination where the order in which items are picked does not matter, as opposed to the permutation.
Definition 2.1: choose
For any integer , we define
Theorem 2.4
Given distinguishable object, there are ways to choose with replacement an ordered sample of objects.
Theorem 2.5
for repetitions of a subexperiment with sample space , the sample space of the sequential experiment has outcomes
Theorem 2.6
The number of observation frequencies for n subexperiments with sample space with 0 appearing times and 1 appearing times is
Theorem 2.7
For repetitions of a subexperiment with sample space the number of length observation sequences with appearing times is
Independent Trials
Theorem 2.8
The probability of failures and successes in independent trials is:
Chapter 3  Discrete Random Variables
Definition  Discrete Variables  A random variable assigns numbers to outcomes in the sample space of an experiment.
A probability model always begins with an experiments. And each of the random variables is directly related to the experiment.
There are 3 types of relationships
 The random variable is the observation
 The random variable is a function of the observation
 The random variable is a function of another random variable.
Throughout most of the book probability is examined in models that assign numbers to the outcomes of the sample space.
When we observe the observation we call it a random variable. Typically in our notation the name of our random variable is a capital letter. e.g. .
The set of possible values for is the range of
Since we often consider multiple variables at a time, we denote the range of a random variable by the letter with a subscript that is the name of the random variable.
Thus, is the range of the random variable , is the range of the random variable and so on…
Definition 3.1
Random Variable consists of an experiment with a probability measure defined on a sample space and a function that assigns a real number to each outcome in the sample space of the experiment.
A note on notation:

On occasion should we need to identify the random variable with respect to a sample outcome, then we define this as . (In other words we define a function which relates the value of X to the outcomes  X is a function of , the outcome)

Sometimes we will write to emphasize that there is a set of sample points for which .
We can then adopt the following shorthand notation:
Definition 3.2
is a discrete random variable if the range of is a countable set
Definition 3.3
Probability Mass Function (PMF)  The probability mass function of the discrete random variable X is
A way of putting this is that for a certain discrete random variable which is described by , then saying is the probability of the event in which that occurs, or in other words it is , where the big is the random variable and simply denotes that it is the PMF function.
Theorem 3.1
For a discrete random variable with a PMF of and range
 For any event , the probability that is in the set is
To put this into words we can say that to calculate the probability of an event, , then it is the sum total probability of all outcomes for which are contained within
Families of Random Variables
 In practical applications, certain types of random variables appear over and over across many experiments.
 In each of these families (types) the PMF of all the random variables will have the same mathematical form
 Differ only in values of one or two parameters
 Depending on family, each PMF will contain only 12 parameters
 If we assign numbers to the parameters we can obtain a specific variable.
 Typical nomenclature for a family consists of the family name followed by one or two parameters in parentheses.
Example: binomial(n, p) refers in general to the family of binomial random variables
Binomial(7, 0.1) refers to the binomial random variable with parameters and
Definition 3.4
X is a Bernoulli (p) random variable if the PMF of X has the form
 The number of successes in a single trial.
Definition 3.5 Geometric (p) Random Variable
X is a geometric (p) random variable if the PMF of X has the form
 Number of trials until (up to and including) the first success
Definition 3.6 Binomial (n, p) Random Variable
X is a binomial (n, p) random variable if the PMF of X has the form
Where and is an integer such that
 the number of successes in n trials
Definition 3.7 Pascal (k, p) Random Variable
X is a pascal (k, p) random variable if the PMF of X has the form:
Where and k is an integer such that
 the number of trials until the success
If then x is geometric.
 Bernoulli (p) = number successes in 1 trial
 Geometric (p) = number of trials until 1st success
 Binomial (n, p) = number of successes in n trials
 Pascal (k, p) = number of trials until k successes
Name of Random Variable  Description of variable  PMF Function 

Bernoulli  number successes in 1 trial  
Geometric  number of trials until 1st success  
Binomial  number of successes in n trials  
Pascal  number of trials until k successes 
Definition 3.8 Discrete Uniform Random Variable
X is a discrete uniform random variable if the PMF of X has the form:
 X is uniformly distributed between and .
Definition 3.9 Poisson () Random Variable
X is a poisson () random variable if the PMF of X has the form
 Call the phenomena of interest an arrival.
 Poisson model specifies an average rate.
 arrivals per second, over a time interval of seconds.
 The number of arrivals has a Poisson PMF with
Section 3.4  Cumulative Distribution Function (CDF)
Like the PMF, the CDF of a random variable X expresses the probability model of an experiment as a mathematical function. The function is the probability for every number .
The cumulative distribution function (CDF) of a random variable, X, is
Theorem 3.2
  The probability which X is less than infinity.
  The probability which X is less than infinity.
 For all
 For and , and arbitrarily small positive number
 for all x such that
Theorem 3.3
For all
In other words, to find the probability of an event between two values, you can simply subtract the values of the CDF at and because it is the sum total probability up to each of those points, subtracting them results in only the probability of values between and .
Definition 3.13 Expected Value
The expected value of is
This is sometimes referred to as the mean of a random variable or function
Theorem 3.4
The Bernoulli random variable has the expected value
Theorem 3.5
The geometric random variable has the expected value
Theorem 3.6
The Poisson () random variable in definition 3.9 has the expected value
Theorem 3.7
1. For the binomial random variable of definition 3.6
2. For the pascal random variable of definition 3.7
3. For the discrete uniform random variable of definition 3.7
Expected Value Table
Function Type  Expected Value 
Bernoulli  
Geometric  
Poisson  
Binomial  
Pascal  
Discrete Uniform 
Theorem 3.8
Perform Bernoulli trials. In each trial let the probability of success be where is a constant and . Let the random variable be the number of successes in trials. As then converges to the PMF of a Poisson random variable
Definition 3.11 Mode
A mode of random variable X is a number satisfying for all x
Definition 3.12 Median
A median of random variable X is a number that satisfies these properties
Functions of a Random Variable
Definition 3.14
Each sample value y of a derived random variable Y is a mathematical function of a sample value x of another random variable X. We adopt the notation to describe the relationship of the two random variables.
Definition 3.9
For a discrete random variable X, the PMF of is
Expected Value of a Derived Random Variable
Theorem 3.10
Given a random variable X with the PMF and derived random variable , the expected value of is
Theorem 3.11
For any random variable X,
Theorem 3.12
For any random variable X,
Variance and Standard Deviation
Definition 3.15 Variance
The variance of a random variable X is:
You might have heard variance before as the square of standard deviation (). Thus follows the next definition.
Definition 3.16 Standard Deviation
The standard deviation of a random variable X is:
Theorem 3.13
In the absence of observations, the minimum mean square error estimate of random variable X is:
Theorem 3.14
Theorem 3.17
For the Random variable, X
a. The nth moment is b. The nth central moment is
Theorem 3.15
Theorem 3.16
Variance Table
Function Type  Variance 
Bernoulli  
Geometric  
Poisson  
Binomial  
Pascal  
Discrete Uniform 
Full Table of Discrete Random Variables
Name of Random Variable  Description of variable  PMF Function  Expected Value  Variance  

Bernoulli  number successes in 1 trial  
Geometric  number of trials until 1st success  
Binomial  number of successes in n trials  
Pascal  number of trials until k successes  
Poisson  Probability of x arrivals in T seconds  
Discrete Uniform  Probability of any value between k and l 
Chapter 4  Continuous Random Variables
Example:
Imagine a circle with an infinite amount of points around the circle. If we were to pick any one point at random. What would the probability of picking that point be?
So what if we ask, what is the probability we choose the point of , or .
Well, it turns out, the answer, . This will lead us into the first section
Section 4.1  Continuous Sample Space
In a continuous sample space, there are an infinite number of points you could choose. But the probability of picking a single point is zero.
Section 4.2  The Cumulative Distribution Function
Definition 4.1  Cumulative Distribution Function (CDF)
Recall this from previous sections. It applies to continuous random variables as the probability of any single event is 0, but for all values up to and including a single value is how we calculate the probability of a continuous function.
Theorem 4.1
For any random variable X
 –
 –
Definition 4.2  Continuous Random Variable
is a continuous random variable if the CDF is a continuous function
Section 4.3  The Probability Density Function
The probability density function (PDF) of a continuous random variable X is:
This function is simply defined as the derivative of the CDF function for a continuous random variable.
Theorem 4.2
For a continuous random variable X with a PDF of
Theorem 4.3
Section 4.4  Expected Values
The expected value of a continuous random variable X is:
Theorem 4.4
The expected value of a function of a random variable X is:
Theorem 4.5
For any random variable
Section 4.5  Families of Continuous Random Variables
Definition 4.5 Uniform Random Variable
X is a uniform (a, b) random variable if the PDF of X is
Where the condition is that the parameter
Theorem 4.6
If X is a uniform (a, b) random variable
 The CDF of X is:
 The expected value of X is
 The variance of X is
Theorem 4.7
Let X be a uniform random variable, where a and b are both integers. Let . Then K is a discrete uniform random variable.
Note that is the “ceiling” of the range of X, or the greatest value in the set
Definition 4.6  Exponential Random Variable
X is an exponential () random variable if the PDF of X is:
Theorem 4.8
If X is an exponential () random variable
 The CDF of X is:
 The expected value of X is
 The variance of X is
Theorem 4.9
If X is an exponential () random variable, then is a geometric (p) random variable with
Definition 4.7  Erlang Random Variable
X is an Erlang () random variable if the PDF of X is:
where the parameter of and the parameter is an integer.
Theorem 4.10
If X is an Erlang () random variable, then
Theorem 4.11
Let denote a Poisson () random variable. For any , the CDF of an Erlang () random variable X, satisfies
Section 4.6  Gaussian Random Variables
Definition 4.8  Gaussian Random Variable
is a Gaussian random variable if the PDF of is:
The parameter can be any real number and the parameter
Theorem 4.12
If X is a Gaussian random variable:
and
Theorem 4.13
If X is a Gaussian random variable, is Gaussian
Definition 4.9  Standard Normal Random Variable
The standard normal random variable is the Gaussian (0, 1) random variable
Definition 4.10  Standard Normal CDF
The CDF of the standard normal random variable Z is:
Theorem 4.14  If X is a Gaussian random variable, then the CDF of X is:
And the probability that X is in the interval of (a, b] (includes b, excludes a):
Theorem 4.15
Definition 4.11  Standard Normal Complementary CDF
The standard normal complementary CDF is:
Another way of looking at is is that
Section 4.7  Delta Functions, Mixed Random Variables
Definition 4.12  Unit Step Function
The unit step function is:
Definition 4.12  Unit Impulse (Delta) Function
Let
Then the unit impulse function is
Now as approaches the delta function for each the area under the curve of the equals 1
Theorem 4.16
For any continuous function g(x)
Theorem 4.17
Chapter 5  Multiviews and Pairs of Random Variables
Pairs of Random Variables
We should consider experiments that produce a collection of random variables .
A pair of random variables and is the same as the two dimensional vector, this way. Since the components of this vector are both random variables we call a random vector.
Definition 5.1  Joint Cumulative Distribution Function (CDF)
The joint cumulative distribution function of random variables and is
So if you can imagine a two dimensional graph that the CDF represents the probability of being within a rectangular region which the point that is picked is x and y
Theorem 5.1
For any pair of random variables
 If then
Theorem 5.2
Section 5.2  Joint Probability Mass Function (PMF)
Definition 5.2  Joint PMF
The joint PMF of discrete random variables and is
Theorem 5.3 For discrete random variables and and any set in the plane, the probability of the event is
Section 5.3  Marginal PMF
Theorem 5.4
For Discrete random variables X and Y with joint PMF
Section 5.4 Joint Probability Density Function
Definition 5.3  Joint PDF
The joint PDF of the continuous random variables and is a function with the property
Theorem 5.5
Theorem 5.6
A joint PDF has the following properties corresponding to the first and second axioms of probability
Theorem 5.7
The probability of the continuous random variables (X, Y) are in A is:
Section 5.5  Marginal PDF
Theorem 5.8
If and are random variables with joint PDF
Section 5.6  Independent Random Variables
Definition 5.4  Independent Random Variables
Random variable and are independent if and only if the following are true:
For discrete random variables:
For continuous random variables:
Section 5.7  Expected Value of a Function of Two Random Variables
Theorem 5.9
For random variables and the expected value of is:
For discrete random variables:
For continuous random variables:
Theorem 5.10
Theorem 5.11
For any two random variables X and Y
Theorem 5.12
The variance of the sum of two random variables is:
Definition 5.5  Covariance
The covariance of two random variables and is:
Definition 5.6  Correlation Coefficient
The correlation coefficient of two random variables X and Y is:
Theorem 5.13
If and , then
Theorem 5.14
Theorem 5.15
If X and Y are random variables such that then:
Definition 5.7  Correlation
The correlation of two variables and is
Theorem 5.16

.

.

If , and
Definition 5.8  Orthogonal Random Variables
Random variables X and Y are orthogonal if
Definition 5.9  Uncorrelated Random Variables
Random variables and are uncorrelated if
Theorem 5.17
For independent random variables X and Y
Section 5.9  Bivariate Gaussian Random Variables
Definition 5.10  Bivariate Gaussian Random Variables
Random variables and have bivariate Gaussian PDF with the parameters , , , , and that satisfying if
Theorem 5.18
If X and Y are bivariate Gaussian random variables in Definition 5.10, X is the Gaussian and the random variable Y is the Gaussian random variable:
Theorem 5.19
Bivariate Gaussian random variables X and Y in Definition 5.10 have the correlational coefficient
Theorem 5.20
Bivariate Gaussian random variables X and Y are uncorrelated if and only if they are independent
Theorem 5.21
If X and Y are bivariate Gaussian random variables with PDF given by definition 5.10 and and are given by the linearly independent equations
Then, and are bivariate Gaussian random variables such that
 where
 where
Section 5.10  Multivariate Probability Models
Definition 5.11  Multivariate Joint CDF
The joint CDF of is:
Definition 5.12  Multivariate Joint PMF
The joint PMF of the discrete random variables is:
Definition 5.13  Multivariate Joint PDF
The joint PDF of the continuous random variables of is:
Theorem 5.22
if are discrete random variables that have a joint PMF
Theorem 5.23
if are continuous random variables that have a joint PDF
Theorem 5.24
The probability of an event A expressed in terms of the random variables is:
Discrete:
Continuous:
Theorem 5.25
For a joint PMF of discrete random variables some marginal PMFs are:
Theorem 5.26
For a joint PDF of discrete random variables some marginal PDFs are:
Definition 5.14  N Independent Random Variables
Random variables are independent if for all
Discrete:
Continuous
Definition 5.15  Independent and Identically Distributed (iid)
are independent and identically distributed if:
Discrete:
Continuous
Chapter 6  Probability Models of Derived Random Variables
Section 6.1  PMF of a Function of Two Discrete Random Variables
Theorem 6.1
For discrete random variables X and Y, the derived random variable has the PMF
Section 6.2  Functions Yielding Continuous Random Variables
Theorem 6.2
If where then has a CDF and PDF of:
CDF  
Theorem 6.3
Given that where
 If is uniform then is uniform
 If is exponential then is exponential
 If is Erlang then is Erlang
 If is Gaussian then is Gaussian
Theorem 6.4
Given that
CDF  
Theorem 6.5
Let be a uniform random variable and let denote a cumulative distribution function with an inverse defined for the random variable has CDF
Section 6.4  Continuous Function of Two Continuous Random Variables
Theorem 6.6
For continuous random variables and , the CDF of is
Theorem 6.7
For continuous random variables and , the CDF of is:
Chapter 7  Conditional Probability Models
Section 7.1  Conditioning a Random Variable by an Event
Definition 7.1  Conditional CDF
Given the event with the conditional cumulative distribution function of X is
Definition 7.2  Conditional PMF Given an Event
Given the event B with , the conditional probability mass function of X is:
Definition 7.3  Conditional PDF Given an Event
Theorem 7.1
For a random variable X and an event with , the conditional PDF of given is:
Discrete
Continuous
Theorem 7.2
For random variable resulting from an experiment with partition
Discrete:
Continuous:
Section 7.2  Conditional Expected Value Given an Event
Theorem 7.3
Discrete X  Continuous X 
for any  for any 
The conditional probability that X is in C:  The conditional probability that X is in the set C is: 
Definition 7.4  Conditional Expected Value
Discrete:
Continuous:
Theorem 7.4
For a random variable resulting from an experiment with partitions
Theorem 7.5
The conditional expected value of given the condition, is:
Discrete:
Continuous:
Section 7.3  Condition Two Random Variables by an Event
Definition 7.6  For discrete random variables and , and event, with , the conditional joint PMF of and given is:
Theorem 7.6
For any event , a region of the plane with
Definition 7.7  Conditional Joint PDF
Given an event with , the conditional joint probability density function of and is:
Theorem 7.7  Conditional Expected Value
For random variables and and an event of nonzero probability, the conditional expected value of given is:
Discrete:
Continuous:
Section 7.4  Condition by a Random Variable
Definition 7.8  Conditional PMF
For any event such that , the conditional PMF of given
Theorem 7.8
For discrete random variables and with joint PMF , and and such that and
Definition 7.9  Conditional PDF
For y such that , the conditional PDF of given is:
Theorem 7.9
For discrete random variables and with joint PMF and and such that and :
Theorem 7.10
For continuous random variables and with joint PDF and and such that and :
Theorem 7.11
If X and Y are independent:
Discrete: , and
Continuous: , and
Section 7.5  Conditional Expected Value Given a Random Variable
Definition 7.10  Conditional Expected Value of a Function
For any , the conditional expected value of given is:
Discrete:
Continuous:
Theorem 7.12
For independent random variables and
 for all
 for all
Definition 7.11  Conditional Expected Value Function
The conditional expected value is a function of random variable Y such that if , then
Theorem 7.13  Iterated Expectation
Theorem 7.14
Theorem 7.15
If and are the bivariate Gaussian random variables in Definition 5.10, the conditional PDF of given is:
where, given , the conditional expected value and variance of are:
Theorem 7.16
If and are the bivariate Gaussian random variables in Definition 5.10, the conditional PDF of given is:
where, given , the conditional expected value and variance of are:
Chapter 9  Sums of Random Variables
Section 9.1  Expected Values of Sums
Theorem 9.1
For any set of random variables , the sum has the expected value:
Theorem 9.2
The variance of is:
Theorem 9.3
When are uncorrelated:
Section 9.2  Moment Generating Functions
Definition 9.1  Moment Generating Function (MGF)
For a random variable , the moment generating function (MGF) of is:
Theorem 9.4
A random variable with MGF has nth moment:
Theorem 9.5
The MGF of is
Section 9.3  MGF of the Sum of Independent Random Variables
Theorem 9.6
For a set of independent random variables , the moment generating function of is:
When are iid, then each with MGF
Theorem 9.7
If are independent Poisson random variables, is a Poisson random variable
Theorem 9.8
The sum of independent Gaussian random variables is a Gaussian random variable
Theorem 9.9
If are iid exponential () random variables, then has the Erlang PDF:
Theorem 9.12  Central Limit Theorem
Given a sequence of iid random variables with expected value and variance , the CDF of
Definition 9.2  Central Limit Theorem Approximation
Let be the sum of iid random variables, each with and . The central limit theorem approximation of the CDF of is:
Chapter 10  The Sample Mean
Definition 10.1  Sample Mean
For iid random variables with PDF , the sample mean of is the random variable:
Theorem 10.1
The sample mean has the expected value and variance
Mean:
Variance:
A note on the sample variance: the the above theorem actually states that the sample mean converges to the expected value as the number of sample, approaches
Section 10.3  The Laws of Large Numbers
Theorem 10.5  Weak Law of Large Numbers (Finite Samples)
For any constant
Theorem 10.6 The Weak Law of Large Numbers (Infinite Samples)
If has finite variance, then, for any constant
Theorem 10.7
as , the relative frequency converges to for any constant