1. CMGPD-LN Methodological Lecture Day 3
Descriptive statistics using TABLE and COLLAPSE

2. Descriptive statistics
There are a number of ways in STATA of transforming the dataset to produce descriptive statistics to be plotted or put into a figure
Slow, manual way
TABULATE
Copy results to Excel, parse, and plot
Not recommended
Transformation to produce counts, averages etc. according to the values of specified variables to use as the basis of plots
TABLE, REPLACE
COLLAPSE
BYSORT combined with EGEN (to be discussed later)

3. Collapsing the data TABLE, REPLACE and COLLAPSE transform the data
For each value of a specified variable, or each combination of values for specified variables, produce a single observation with summary statistics of other specified values
These summary statistics can be counts, sums, means, etc.

4. COLLAPSE Start with a hypothetical dataset +----------------+
| x1 x y |
| |
1. | |
2. | |
3. | |
4. | |
5. | |
6. | |
7. | |
8. | |
9. | |
10. | |
Replace the dataset with one that for each combination of x1 and x2, contains the mean of y
. collapse y, by(x1 x2)
. list
| x1 x y |
| |
1. | |
2. | |
3. | |
4. | |

5. Or count the numbers of records for each unique combination of x1 and x2
. collapse (count) y, by(x1 x2)
. list
| x1 x2 y |
| |
1. | |
2. | |
3. | |
4. | |
Or both at the same time, creating count and average simultaneously. ‘avgy=‘ tells it to create a new variable name.
. collapse (count) y (mean) avgy=y, by(x1 x2)
. list
| x1 x2 y avgy |
| |
1. | |
2. | |
3. | |
4. | |

6. TABLE, REPLACE
Can achieve the same thing with TABLE, REPLACE, though the resulting variable names are a bit cryptic
. table x1 x2, contents(count y mean y) replace
| x2
x1 |
1 | | |
2 | |
. list
| x1 x2 table1 table2 |
| |
1. | |
2. | |
3. | |
4. | | .

7. Observations by year
The easy way to get a figure for numbers of observations by register year is to use histogram. histogram YEAR, discrete frequency ytitle("Observations") xtitle("Year") xlabel(1750(25)1900) To force a monochromatic color scheme, we can add scheme(s1mono) To override the default numeric format of the vertical axis labels, we can add ylabel(,format(“%5.0f”)) histogram YEAR, discrete frequency ytitle("Observations") xtitle("Year") xlabel(1750(25)1900) ylabel(,format(%5.0f)) scheme(s1mono)

10. Observations by year
We could do the same thing with table to prepare the dataset, and then twoway bar. table YEAR, contents(freq) replace twoway bar table1 YEAR, scheme(s1mono) xlabel(1750(25)1900) ytitle("Number of observations") Or if we want to do it as a scatter plot… twoway scatter table1 YEAR, scheme(s1mono) xlabel(1750(25)1900) ytitle("Number of observations")

13. Registers by year
The number of available registers varies year by year.
This accounts for some of the year to year fluctuation in numbers of observations
In some cases, may also account for some of the year to year fluctuation in other summary values
We can do a year by year count of the number of available registers easily enough

14. Registers by year table YEAR DATASET, replace table YEAR, replace
twoway bar table1 YEAR, scheme(s1mono) ytitle("Registers")
Let’s use of angle and labsize on xlabel to label each register year individually
twoway bar table1 YEAR, scheme(s1mono) ytitle("Registers") xlabel(1750(3)1909,angle(vertical) labsize(vsmall))
Note that coverage is much more sparse before 1789.
Some years (1810) are missing an especially large number of registers
No registers at all from 1888 to 1903

17. Population by age group
Let’s use TABLE to look at the distribution of the population by age group keep if PRESENT & AGE >= 1 & AGE <= 75, clear recode AGE_IN_SUI 1/15=1 16/55=16 56/75=56, generate(AGE_GROUP) tab AGE_GROUP SEX if SEX >= 1, col row table AGE_GROUP SEX if SEX >= 1, col row

18. RECODE of |
AGE_IN_SUI |
(Age in | Sex
Sui) | Female Male | Total
1 | 36, ,332 | 270,632
| |
| |
16 | 393, ,381 | 894,358
| |
| |
56 | 97, ,716 | 196,049
| |
| |
Total | 527, ,429 | 1,361,039
| |
| |

19. -------------------------------------
RECODE of |
AGE_IN_SU |
I (Age in | Sex
Sui) | Female Male Total
1 | 36, , ,632
16 | 393, , ,358
56 | 97, , ,049 |
Total | 527, ,

20. Counts, averages, proportions by age and time
There are a variety of options for collapsing observations to produce counts, proportions, averages, etc. by year, age, etc.
One simple approach is the table command, combined with the replace option
This replaces the dataset in memory with a ‘collapsed’ version
Values in the ‘collapsed’ version can be plotted with twoway bar etc.

21. table AGE_GROUP SEX if SEX >= 1, by(YEAR) replace
table AGE_GROUP SEX if SEX >= 1, by(YEAR) replace * Entries created for totals have missing values for AGE_GROUP drop if AGE_GROUP == . reshape wide table1, i(YEAR SEX) j(AGE_GROUP) * Also need to remove newly created totals with missing values for SEX drop if SEX == . reshape wide table11 table116 table156, i(YEAR) j(SEX) generate male_proportion_16_55 = table1162/(table112+table1162+table1562) twoway bar male_proportion_16_55 YEAR, ytitle("Proportion of males who are 16 to 55 sui") xtitle("Year") ylabel(0(0.1)1) scheme(s1mono) generate male_dependency_ratio = (table112+table1562)/(table1162) twoway bar male_dependency_ratio YEAR, ytitle("Male dependency ratio (( )/(16-55) ") xtitle("Year") ylabel(0(0.1)1) scheme(s1mono) generate child_sex_ratio = table112/table111 twoway bar child_sex_ratio YEAR, ytitle("Ratio of males to females aged 1-15 sui") xtitle("Year") scheme(s1mono) yscale(log) ylabel( )

22. Reshape
Notice that TABLE (and COLLAPSE) will produce one observation for each combination of YEAR, age_group, and SEX
50*3*2=300 observations (approximately)
299 in reality because one cell is empty
We would like one observation per year
In order to carry out calculations
Use reshape to convert to one observation per combination of YEAR and SEX, with three variables, one each for each of the age groups
Use reshape again to convert to one observation per YEAR, with six variables per observation, one for each combination of SEX and age_group
Can calculate dependency ratios, sex ratios etc. from these numbers

26. Proportions/means Proportion ever married by year
We can also calculate means of specified variables by YEAR, AGE_IN_SUI, or other variables of interest
use "C:\Users\Cameron Campbe\Documents\Baqi\CMGPD-LN from ICPSR\ICPSR_27063\DS0001\ Data.dta" if PRESENT & AGE >= 16 & AGE <= 50 & SEX == 2 & MARITAL_STATUS >= 0, clear
recode AGE_IN_SUI 16/30=16 31/40=31 41/50=41, generate(age_group)
generate ever_married = MARITAL_STATUS != 2
table YEAR age_group, contents(mean ever_married) replace
twoway bar table1 YEAR if age_group == 16,ylabel(0(0.1)1) ytitle("Proportion of men ever married") xtitle("Year") scheme(s1mono)
twoway bar table1 YEAR if age_group == 31,ylabel(0(0.1)1) ytitle("Proportion of men ever married") xtitle("Year") scheme(s1mono)

29. Proportion married by age
use "C:\Users\Cameron Campbe\Documents\Baqi\CMGPD-LN from ICPSR\ICPSR_27063\DS0001\ Data.dta" if PRESENT & AGE >= 1 & AGE <= 50 & SEX == 2 & MARITAL_STATUS >= 0, clear generate ever_married = MARITAL_STATUS != 2 table AGE_IN_SUI, contents(mean ever_married) replace twoway bar table1 AGE_IN_SUI, ylabel(0(0.10)1) ytitle("Proportion of males ever married") xtitle("Age in sui") scheme(s1mono)

31. Multiple trends in the same graph
keep if SEX == 2 & PRESENT & BIRTHYEAR >= 1750 & BIRTHYEAR <= 1900 keep if MARITAL_STATUS > 0 keep if AGE_IN_SUI >= 11 & AGE_IN_SUI <= 40 recode AGE_IN_SUI 11/15=11 16/20=16 21/25=21 26/30=26 31/35=31 36/40=36, generate(age_group) generate ever_married = MARITAL_STATUS != 2 table BIRTHYEAR age_group, contents(mean ever_married) replace twoway line table1 BIRTHYEAR if age_group == 11 || line table1 BIRTHYEAR if age_group == 16 || line table1 BIRTHYEAR if age_group == 21 || line table1 BIRTHYEAR if age_group == 26 || line table1 BIRTHYEAR if age_group == 31 || line table1 BIRTHYEAR if age_group == 36 || ,scheme(s1mono) legend(order(1 "11-15 sui" 2 "16-20 sui" 3 "21-25 sui" 4 "26-30 sui" 5 "31-35 sui" 6 "36-40 sui")) ytitle("Proportion of males ever married")

33. Using COLLAPSE
keep if PRESENT & SEX == 2 & AGE_IN_SUI > 1 & AGE_IN_SUI <= 60
mvdecode _all, mv( )
generate MARRIED = MARITAL_STATUS == 1
By default, collapse will create variables of the same name containing means
collapse MARRIED SON_COUNT DAUGHTER_COUNT FATHER_ALIVE MOTHER_ALIVE BROTHER_COUNT, by(AGE_IN_SUI)
Notice use of legend to specify a label for each of the 5 lines
twoway line FATHER_ALIVE MOTHER_ALIVE MARRIED SON_COUNT BROTHER_COUNT AGE_IN_SUI, scheme(s1mono) legend(order(1 "Father alive" 2 "Mother alive" 3 "Wife alive" 4 "Sons ever born" 5 "Brothers alive")) ytitle("Mean") lpattern(solid solid dash dot dash_dot)

35. Calculating rates
Calculation of demographic rates by age and so forth is straightforward, using the AT_RISK_* and NEXT_* flag variables.
Let’s calculate and compare probability of marriage in the next three years by age, for men and women
keep if AT_RISK_MARRY == 1 & SEX > 0 & AGE_IN_SUI > 0 & AGE_IN_SUI <= 30
collapse NEXT_MARRY, by(AGE_IN_SUI SEX)
twoway line NEXT_MARRY AGE_IN_SUI if SEX == 1 || line NEXT_MARRY AGE_IN_SUI if SEX == 2 || , legend(order(1 "Female" 2 "Male")) scheme(s1mono)