Probability and Stochastic Processes
EL6303, Fall 2013
Prereq: Graduate status and MA 3012. *Online version available.
Professor Ted Rappaport
Wednesdays – 3:00 PM – 5.30 PMx
Room: JAB474
Office Hours: Wednesdays, 11 am – 12 noon or by appointment, 370 Jay Street, 9th Fl, Brooklyn, NY 11201
Required Text: Probability, Random Variables and Stochastic Processes by Athanasios Papoulis and S. Unnikrishna Pillai
Announcements
Posted on December 12, 2013
Lecture notes from 12/11/13
Pillai Lecture notes – 2
Pillai Lecture notes – 1
Posted on December 12, 2013
Final exam room assignments.
Date: 12/18/2013
Time: 3:005:30pm
Room Assignments by Lastname:
AM: JAB474
NWang: RH315
WuZhu: JAB678
Posted on December 4, 2013
Useful laplace and fourier table, this will be provided on the final exam, along with inside book covers.
LaplaceAndFourier.pdf (Filesize: 865 kb)
Approximately 1/3 of the final exam will cover material after exam 2 and 2/3 of the final exam will cover material up to and including exam 2. Three 8.5 x 11 Crib sheets may be used by students; this exam is closed book and notes. The final exam date is December 18, 2013 more details to come.
Posted on November 10, 2013
Exam 2 to be on November 13, 2013, 3pm. GTA George MacCartney shall make the exam room assignments. You may use 2 doublesided 8.5X11” crib sheet. I shall provide tables from the inside book covers. No notes, no calculators, no cellphones and no sharing items allowed.
HW 8 â€“ everyone receives a 100 for this homework and the solution shall be posted by Nov. 11, 2013, before the exam. This material will be covered on Exam 2.
Posted on September 25, 2013
Please Note to the class: HW 4 is due BEFORE 2pm on Monday 9/30, and the material will also be covered as part of Exam 1. Submit to the GTA George MacCartney by 2pm in room 9.009 at the end of the help session or submit by uploading on blackboard a scanned PDF following the homework guidelines.
Posted on September 20, 2013
The first exam will be held on Oct. 2, 2013 in room RH116, 3:005:30pm. The exam is closed book, closed notes,no cellphones, no calculators of any kind. Only one double sided 8.5×11” crib sheet allowed. Please bring nothing more than pencils/pens and your crib sheet.
Welcome from Prof. Rappaport
Welcome to all of the new ECE graduate students at NYU Tandon! I am very excited to be teaching EL 6303, “Probability and Stochastic Processes“, the most important core course in ECE, and I look forward to having you in class!
This course is the fundamental core course for all degrees in ECE, and you must master this material to succeed in graduate school, in research, and in life. I encourage students to buy the book early, and begin reading the book (4th edition US edition) *before* our first day of class. I will hold weekly help sessions on Monday and Friday afternoons, and we will work lots of problems, I will hold extra office hours, and we will have TAs to help everyone master the material, but you will have to work hard. This is a challenging and rigorous class; it is also required class for your program. Come prepared to work and to learn this wonderful field!
Weekly Help Sessions And Resources
To ensure knowledge and learning, will be held on Friday’s and Monday’s every week. Be sure to attend these help sessions to be sure you are learning the material. You want to come to these help sessions as many problems will be worked for practice and understanding!
Help Sessions : 2 Metrotech Center Classroom 9.009, Mondays 122PM and Fridays 12 – 2 PM. (There will be no help session on Friday October 4th, 2013.)
In addition to this website please see blackboard for additional resources.
Link to course blackboard page: https://blackboard.poly.edu/
See below for useful notes from the Fall 2011 El6303 lectures:
View EL6303 Lecture notes
Graduate Teaching Assistant (GTA)
George Maccartney’s Email: gmac@nyu.edu.
Office Hours: (Cube 9.120I) 9th Floor of 2 MetroTech, 12pm2pm on Tuesdays and Thursdays.
Course Grader
For all inquires regarding the homework grading, please contact the course GTA, George Maccartney. Your course grader is Yuan Zhang.
Yuan Zhang’s Email: yz1484@nyu.edu
Exam Solutions
Exam #1:
Please note that the odd answers include the solutions for solving the exam.
EL6303 Prof. Rappaport Exam one Answer Keys (Even Seats)
EL6303 Prof. Rappaport Exam one Solutions (Odd seat numbers)
Exam #2:
The following documents include the answer key for the Odd and Even exams, the worked out solutions by the professor, as well as extra notes from the help session.
Exam2_Answer_Key.pdf
EL6303_Fall2013_Exam2_Odd_Solution.pdf
EL6303_Fall2013_Exam2_Even_Solution.pdf
Final Exam:
EL6303 Final Exam Set 1 Solution Dec 26 5pm.pdf
Homework
Homework submission guidelines.
1. All work on 8.5×11 sheets
2. Cover Sheet with full name registered with NYU Tandon (Last, First) (NO NICK NAMES), followed by NYU ID #. Example:
Homework #1
MacCartney, George
NYU ID: 0500000
3. Staple all sheets together
For online students in section EL6303.1129 follow these additional guidelines:
4. Upload your homework under the assignments section in blackboard
5. Do not submit photos from a camera or phone, the homework must be scanned in.
6. Only submit one PDF file containing all of your work.
7. The title of the file should follow this example: LastName_NYUID_EL6303_HWX:
MacCartney_0500000_EL6303_HW2.pdf
8. The deadline for submissions is 3:00 pm every Wednesday. Homework will not be accepted after that.
Assignment 1: Assigned 9/4/13; Due 9/11/13
Students may choose to work either Problem 9 or problem 10, you don’t have to work both of these if you do not wish to.
View HW 1 Due 9/11/13
View HW 1 solutions
Assignment 2: Assigned 9/11/13; Due 9/18/13
View HW 2 Due 9/18/13
View HW 2 solutions
Assignment 3: Assigned 9/18/13; Due 9/25/13
View HW 3 Due 9/25/13
View HW 3 solutions
Assignment 4: Assigned 9/25/13; Due 9/30/13 at 2 P.M.
This material will be covered on the exam, submit to the GTA George MacCartney by 2pm in room 9.009 at the end of the help session or submit by uploading on blackboard a scanned PDF following the homework guidelines.
View HW 4 Due 9/30/13 at 2 P.M.
View HW 4 solutions
Assignment 5: Assigned 10/9/13; Due 10/23/2013 at 3pm
View HW 5 Due 10/23/13 at 3 P.M
View HW 5 solutions
Assignment 6: Assigned 10/23/13; Due 10/30/2013 at 3pm
View HW 6 Due 10/30/13 at 3 P.M
View HW 6 solutions
Assignment 7: Assigned 10/30/13; Due 11/06/2013 at 3pm
View HW 7 Due 11/06/13 at 3 P.M
View HW 7 solutions
Assignment 8: Assigned 11/06/13; Due 11/13/2013 at 3pm
View HW 8 Due 11/13/13 at 3 P.M
View HW 8 solutions
Assignment 9: Assigned 11/13/13; Due 11/27/2013 at 3pm
View HW 9 Due 11/27/13 at 3 P.M
View HW 9 solutions
Assignment 10: Assigned 11/27/13; Due 12/04/2013 at 3pm
View HW 10 Due 12/04/13 at 3 P.M
View HW 10 solutions
View HW10 Extra Credit solution
View HW10 Q2 additional solution
Grading
Homework must be turned in personally to Prof. Rappaport at the beginning of class. Everyone must do his or her own work without outside assistance.
Grading for the course will be based on:
Homework: 15%
Exam 1 Oct. 2, 2013: 25%
Exam 2 Nov. 13, 2013: 25%
Final exam: 35%
Lecture Schedule
Date  Topic  Reading Assignments  Important Events in Class 

9/4  Induction, Deduction, Relative Frequency, Sets, Repeated Trials, Bernoulli Trials  Pages 1 70  
9/11  Conditional Probability, Bayes’ Theorem and Independent events; Random Variables; Probability Distribution and density functions; Continuous and discrete random variables.  Pages 72 105  HW 1 is due, assigned 9/4/13. 
9/18  Functions of one Random Variable and their distributions; Expected value and Variances of a Random Variable.  Pages 106151  HW 2 is due assigned 9/11/13. 
9/25  Asymptotic Approximations, Characteristic Functions, Moment Generating Functions, and Higher Order Moments.  Pages 151165  HW 3 due assigned9/18/13. 
9/30  This material will be covered on the exam, submit to the GTA George MacCartney by 2pm in room 9.009 at the end of the help session or submit by uploading on blackboard a scanned PDF following the homework guidelines.  HW 4 due assigned9/25/13.  
10/2  Inclass exam, one 8.5 X 11″ crib sheet allowed  The exam is closed book, closed notes, no calculators of any kind. Only one double sided 8.5×11” crib sheet allowed.  
10/9  Joint distribution and density function of Two Random Variables; Independent Random Variables; Jointly Gaussian Random Variables; One function of Two Random Variables and its distribution; Discrete Random Variables and their Functions.  Pages 169196  HW 5 assigned and due 10/23/13. 
10/23  Probability density functions (pdf) of One and Two Function(s) of Two Random Variables – Sum, Difference, Ratio, Product, Magnitude, Phase, Minimum, Maximum, Min/Max etc. – and their joint density functions; the Jacobean, Jointly distributed Discrete Random Variables and their Functions. Linear functions of Gaussian Random Variables and their joint density functions. Covariance, Correlation, Orthogonality; Uncorrelated and Independent Random Variables; Joint characteristic function and higher order Moments.  Pages 197219  HW 5 due.
Supplement for students regarding Leibniz Rule / Theorem: 
10/30  Conditional distributions and conditional density functions; Conditional Mean and Variance; Conditional Gaussian Random Variables and their Mean and Variances. Stochastic Processes; Concept of Stationarity; Strict Sense Stationary (SSS) and Wide Sense Stationary (WSS) Stochastic Processes; Auto correlation function and its properties;  Pages 220235
Pages 373393 
HW 6 due. 
11/06  Stochastic Processes; Concept of Stationarity; Strict Sense Stationary (SSS) and Wide Sense Stationary (WSS) Stochastic Processes; Auto correlation function and its properties; Examples. Stationary Gaussian Process input to Memoryless systems. Discrete Time Processes.  Pages 373393  HW 7 due. 
11/13  Inclass exam, two 8.5 X 11″ crib sheet allowed  HW 8 due.
The exam is closed book, closed notes, no calculators of any kind. Only two 8.5×11” crib sheet allowed. 

11/20  Systems with stochastic inputs, memoryless systems, strict sense stationary, square law detection, linear systems, output correlation, Response to white noise, differentiators, power spectral density for WSS, Line spectra, moving average filter, differentiations, Hilbert Transform, WienerKhinchin Theorem, Least Mean Square Error, Cross Correlation  Pages 393430  
11/27  Linear Systems, Factorization for Partial Fraction Expansion, Discrete Time systems, finite order systems, state variables, Fourier Series, KarhunenLoeve expansion, Spectral Representation of Random Processes  Pages 499521  HW 9 due. 
12/4  Mean square estimation, Interpolation, Markov sequences, Wiener Filter, Wiener Hopf equation, Predictors  Pages 580605  HW 10 due 
12/11  Approximately 1/3 of the final exam will cover material after exam 2 and 2/3 of the final exam will cover material up to and including exam 2. Three 8.5 x 11 Crib sheets may be used by students; this exam is closed book and notes. The final exam date is December 18, 2013 more details to come.  Course Review 
Course Outline
Continuous and discrete random variables and their joint Probability Distribution and density functions; Functions of one Random Variable and their distributions Independent Random Variables and conditional distributions; One Function of One and Two Random Variables and Two Functions of Two Random variables and their joint density functions; Jointly distributed Discrete Random Variables and their Functions; Characteristic Functions and Higher Order Moments; Covariance, Correlation, Orthogonality; Jointly Gaussian Random Variables; Linear functions of Gaussian Random Variables and their joint density functions. Stochastic Processes and the concept of Stationarity; Strict Sense Stationary (SSS) and Wide Sense Stationary (WSS) Processes; Auto correlation function and its properties; Poisson Processes and Wiener processes; Stochastic Inputs to Linear TimeInvariant (LTI) Systems and their inputoutput Autocorrelations; InputOutput Power Spectrum for Linear Systems with Stochastic Inputs; Minimum Mean Square Error Estimation(MMSE) and Orthogonality Principle; Auto Regressive Moving Average (ARMA) Processes and their power spectra.
1. Conditional Probability, Bayes’ Theorem and Independent events; Random
variables; Probability Distribution and density functions; Continuous and
discrete random variables.
2. Functions of one Random Variable and their distributions; Expected value and
Variances of a Random Variable; Characteristic Functions, Moment Generating
Functions, and Higher Order Moments.
3. Joint distribution and density function of Two Random Variables; Independent
Random Variables; One function of Two Random Variables and its distribution;
Discrete Random Variables and their Functions.
4. Probability density functions (pdf) of One and Two Function(s) of Two Random
Variables – Sum, Difference, Ratio, Product, Magnitude, Phase, Minimum,
Maximum, Min/Max etc. – and their joint density functions; Jointly distributed
Discrete Random Variables and their Functions
5. Covariance, Correlation, Orthogonality; Uncorrelated and Independent Random
Variables; Joint characteristic function and higher order Moments.
6. Jointly Gaussian Random Variables; Linear functions of Gaussian Random
Variables and their joint density functions
7. Conditional distributions and conditional density functions; Conditional Mean
and Variance; Conditional Gaussian Random Variables and their Mean and
Variances
8. Midterm Exam
9. Stochastic Processes; Concept of Stationarity; Strict Sense Stationary (SSS) and
Wide Sense Stationary (WSS) Stochastic Processes; Auto correlation function and
its properties; Examples.
10. Poisson Processes: Axiomatic development; Probability Distributions of First and
Second arrivals; InterArrival distributions; Distributions of Poisson Arrivals
within the InterArrival/Departure Interval of another Independent Poisson
Process(Geometric Distribution); Wiener Processes; Compound Poisson
Processes.
11. Wide Sense Stationary Processes and their Autocorrelation Functions; Stationary
Gaussian Process input to Memoryless systems. Discrete Time Processes.
12. Stochastic Inputs to Linear TimeInvariant (LTI) Systems; InputOutput
Autocorrelations; Differentiators; Probability of ZeroCrossing for Stationary
Gaussian Processes.
13. Stationary Processes and Power Spectrum; InputOutput Power Spectrum for
Linear Systems with Stochastic Inputs; Quadrature Filtering (Hilbert Transform)
and Single Sideband Modulation.
14. Linear Estimation; Minimum Mean Square Error Estimation(MMSE) and
Orthogonality Principle ; Equivalence of Best Estimator and best Linear
Estimator for Discrete Gaussian Processes; Auto Regressive (AR), Moving
Average (MA), and Auto Regressive Moving Average (ARMA) Processes and their
power spectra.
15.Final Exam
CONTACT
Office Hours: Wednesdays, 11 am – 12 noon or by appointment, 2 Metrotech Center 9th floor
Required Text: Probability, Random Variables and Stochastic Processes by Athanasios Papoulis and S. Unnikrishna Pillai
QUICK LINKS
NYUSIM: The 5G Simulator
NYU WIRELESS Newletter
Publications Library