Panel data

In statistics and econometrics, the term panel data (or longitudinal data^[1][2]) refers to multi-dimensional data frequently involving measurements over time. Panel data contain observations of multiple phenomena obtained over multiple time periods for the same firms or individuals.

Time series and cross-sectional data can be thought of as special cases of panel data that are in one dimension only (one panel member or individual for the former, one time point for the latter).

A study that uses panel data is called a longitudinal study or panel study.

Example

balanced panel:		unbalanced panel:
${\begin{matrix}\mathrm {person} &\mathrm {year} &\mathrm {income} &\mathrm {age} &\mathrm {sex} \\1&2001&1300&27&1\\1&2002&1600&28&1\\1&2003&2000&29&1\\2&2001&2000&38&2\\2&2002&2300&39&2\\2&2003&2400&40&2\end{matrix}}$		${\begin{matrix}\mathrm {person} &\mathrm {year} &\mathrm {income} &\mathrm {age} &\mathrm {sex} \\1&2001&1600&23&1\\1&2002&1500&24&1\\2&2001&1900&41&2\\2&2002&2000&42&2\\2&2003&2100&43&2\\3&2002&3300&34&1\end{matrix}}$

In the example above, two data sets with a panel structure are shown. Individual characteristics (income, age, sex) are collected for different persons and different years. In the left data set two persons (1, 2) are observed over three years (2001, 2002, 2003). Because each person is observed every year, the left-hand data set is called a balanced panel, whereas the data set on the right hand is called an unbalanced panel, since person 1 is not observed in year 2003 and person 3 is not observed in 2003 or 2001. This specific structure these data sets are in is called long format where one row holds one observation per time. Another way to structure panel data would be the wide format where one row represents one observational unit for all points in time (for the example, the wide format would have only two (left example) or three (right example) rows of data with additional columns for each time-varying variable (income, age). Representing panel data in long format is much more common than using the wide format.

Analysis of panel data

A panel has the form

$X_{it},\;i=1,\dots ,N\;t=1,\dots ,T,$
where $i$ is the individual dimension and $t$ is the time dimension. A general panel data regression model is written as $y_{it}=\alpha +\beta 'X_{it}+u_{it}.$ Different assumptions can be made on the precise structure of this general model. Two important models are the fixed effects model and the random effects model. The fixed effects model is denoted as

$y_{it}=\alpha +\beta 'X_{it}+u_{it},$

$u_{it}=\mu _{i}+\nu _{it}.$

$\mu _{i}$ are individual-specific, time-invariant effects (for example in a panel of countries this could include geography, climate etc.) and because we assume they are fixed over time, this is called the fixed-effects model. The random effects model assumes in addition that

$\mu _{i}\sim {\text{i.i.d.}}N(0,\sigma _{\mu }^{2})$

and

$\nu _{it}\sim {\text{i.i.d.}}N(0,\sigma _{\nu }^{2}),$

that is, the two error components are independent from each other.

Dynamic Panel Data

Dynamic panel data describe the case where a lag of the dependent variable is used as regressor:

${\displaystyle y_{it}=\alpha +\beta 'X_{it}+\gamma y_{it-1}+u_{it},}$

The presence of the lagged dependent variable introduces endogeneity, so estimators like Arellano–Bond estimator or Blundell and Bond that use instruments must be used.

Data sets which have a panel design

• Russia Longitudinal Monitoring Survey (RLMS)
• German Socio-Economic Panel (SOEP)
• Household, Income and Labour Dynamics in Australia Survey (HILDA)
• British Household Panel Survey (BHPS)
• Survey of Family Income and Employment (SoFIE)
• Survey of Income and Program Participation (SIPP)
• Lifelong Labour Market Database (LLMDB)
• Panel Study of Income Dynamics (PSID)
• Korean Labor and Income Panel Study (KLIPS)
• Chinese Family Panel Studies (CFPS)
• German Family Panel (pairfam)
• National Longitudinal Surveys (NLSY)
• Labour Force Survey (LFS)
• Korean Youth Panel (YP)
• Korean Longitudinal Study of Aging (KLoSA)

Data sets which have a multi-dimensional panel design

See also

• Panel study

Notes

1. Diggle, Peter J.; Heagerty, Patrick; Liang, Kung-Yee; Zeger, Scott L. (2002). Analysis of Longitudinal Data (2nd ed.). Oxford University Press. p. 2. ISBN 0-19-852484-6.
2. Fitzmaurice, Garrett M.; Laird, Nan M.; Ware, James H. (2004). Applied Longitudinal Analysis. Hoboken: John Wiley & Sons. p. 2. ISBN 0-471-21487-6.

References

• Baltagi, Badi H. (2008). Econometric Analysis of Panel Data (Fourth ed.). Chichester: John Wiley & Sons. ISBN 978-0-470-51886-1.
• Davies, A.; Lahiri, K. (1995). "A New Framework for Testing Rationality and Measuring Aggregate Shocks Using Panel Data". Journal of Econometrics. 68 (1): 205–227. doi:10.1016/0304-4076(94)01649-K.
• Davies, A.; Lahiri, K. (2000). "Re-examining the Rational Expectations Hypothesis Using Panel Data on Multi-Period Forecasts". Analysis of Panels and Limited Dependent Variable Models. Cambridge: Cambridge University Press. pp. 226–254. ISBN 0-521-63169-6.
• Frees, E. (2004). Longitudinal and Panel Data: Analysis and Applications in the Social Sciences. New York: Cambridge University Press. ISBN 0-521-82828-7.
• Hsiao, Cheng (2003). Analysis of Panel Data (Second ed.). New York: Cambridge University Press. ISBN 0-521-52271-4.

External links

• PSID
• KLIPS
• pairfam
• Korea Employment Survey