S&P 500 Daily Time Series since 1927 (GitHub fja05680) (Date)

Rows: 23,104
Columns: 7

S&P 500 daily OHLCV time series dating back to 1927, sourced from Yahoo Finance. One of the longest freely available daily price records for the index.

AI analysis

Dataset Analysis: S&P 500 Daily Time Series (1927–2019)

1. What this dataset represents and its value for correlation studies

This dataset is a long-run daily OHLCV (Open, High, Low, Close, Volume) time series for the S&P 500 index, spanning nearly 92 years (1927-12-30 to 2019-12-23) across 23,104 trading days. It's sourced via the GitHub repository fja05680/dow-sp500-100-years, which aggregates Yahoo Finance data — a publisher pattern common among independent quant/finance hobbyist archives rather than an official exchange feed. This matters for correlation work: the sheer time depth (covering the Great Depression, WWII, postwar expansion, stagflation, the 1987 crash, dot-com bubble, 2008 financial crisis, and the 2010s bull run) makes it one of the few freely available datasets long enough to study cross-regime relationships — e.g., how equities correlate with commodities, rates, or currencies differently across monetary regimes (gold standard vs. fiat, Bretton Woods vs. floating rates). Note the dataset stops at end of 2019, so it excludes COVID-era and post-2020 volatility — any correlation study needs a supplementary source for recent years.

2. Data quality observations

Data quality is notably strong on the surface: zero nulls across all 7 columns and 23,104 rows, and the Date join key shows full distinct-value coverage (23,104 distinct dates matching row count), implying no duplicate dates within the ingested set (though the duplicate-row metric itself is still pending Phase D Parquet recomputation, so a final dedup confirmation is outstanding). One quality flag worth investigating: Volume's minimum value is 0, which is suspicious for a major index — zero-volume trading days are likely data artifacts from early decades (pre-electronic reporting) or holiday placeholder rows rather than genuine no-trade days, and should be filtered or flagged before using Volume in analysis. The distinct-value counts for High/Low/Open/Close/Adj Close (~15,584–15,612 out of 23,104 rows) are also expected given repeated round-number prices in low-price early decades.

3. Key columns and their distributions

Price columns (Open, High, Low, Close, Adj Close) are nearly identical in distribution — mean ≈ 461–464, median ≈ 98–100, σ ≈ 677–685 — with a large mean-median gap and skewness around 1.75–1.76, reflecting the exponential-like growth of the index over time (prices under $30 for decades before compounding into the thousands). Q1 (~23.68) to Q3 (~750–755) spans two orders of magnitude, and outlier counts (1,466–1,485 per price column, ~6.3–6.4% of rows) almost certainly represent the modern high-price era (2000s–2010s) rather than anomalies — a reminder that simple IQR-based outlier detection is misleading on non-stationary financial series and log-transformation or returns-based analysis is preferable. Volume is even more skewed (skew=2.2, mean $7.3B$ vs. median $16.65M$ — a 44x gap), with 4,628 outliers (~20% of rows), consistent with the well-documented explosion in trading volume/liquidity from the 1990s onward (decimalization, algo trading, ETFs) relative to the illustrative low-volume early 20th century.

4. Recommended join key columns

Date is the only and obvious join key — it's fully populated, fully distinct, and daily-grained. For cross-dataset joins, standardize on trading-day alignment (exclude weekends/holidays) and consider a calendar-normalization step (e.g., converting to ISO date strings) since many macro datasets report monthly or weekly rather than daily. When joining to lower-frequency data, use forward-fill or period-end alignment; when joining to higher-frequency data (intraday), the Date field will need aggregation.

5. Suggested pairings for correlation discovery

Strong candidates include: (a) macroeconomic indicators — CPI/inflation, Fed Funds rate, unemployment, GDP (FRED) to study equity-inflation and equity-rate regimes across a full century; (b) other asset classes — gold/commodity prices, US Treasury yields, and the US Dollar Index, to examine flight-to-safety and risk-on/risk-off dynamics historically; (c) the Dow Jones Industrial Average (likely available in the same GitHub repo given its naming) for direct index-comparison and divergence analysis; (d) volatility/crisis event datasets (e.g., NBER recession dates, banking crisis timelines) to correlate Volume/price-range spikes with known shocks; and (e) international equity indices (FTSE, Nikkei) for global market co-movement studies, particularly valuable given this dataset's rare coverage of the pre-1970s period that most free datasets lack.

Columns

Date (date)
High (decimal)
Low (decimal)
Open (decimal)
Close (decimal)
Volume (integer)
Adj Close (decimal)