Trove newspapers: data dashboard¶
Trove's collection of digitised newspapers is always changing – new articles and newspapers are being added by the NLA, while Trove users are busy correcting OCRd text and adding tags and comments. The search you run today might produce different results than the same query did a month ago, a year ago, or ten years ago. Researchers need to understand how these changes affect the queries they make, the results they find, and the arguments they construct. This dashboard helps researchers understand the context of their queries by presenting a snapshot of Trove's digitised newspapers, based on weekly data harvests. It shows Trove's current make up from a number of angles, as well as highlighting recent changes. It is updated every Sunday.
Total articles and user activity¶
| Latest data harvest | 13 November 2022 |
|---|---|
| First data harvest | 24 April 2022 |
| Total harvests | 30 |
| Data | https://github.com/wragge/trove-newspaper-totals/blob/master/data/total_articles_by_activity.csv |
As of 13 November 2022 there are 236,213,738 digitised newspaper articles in Trove.
| Current total | Change since 06 Nov 2022 | Change since 16 Oct 2022 | Change since 24 Apr 2022 | Percentage of total | |
|---|---|---|---|---|---|
| All articles | 236,213,738 | 49,544 | 269,713 | 1,701,648 | 100.00% |
| Articles with corrections | 14,348,436 | 15,786 | 67,088 | 492,897 | 6.07% |
| Articles with tags | 4,585,997 | 4,903 | 22,121 | 183,003 | 1.94% |
| Articles with comments | 251,461 | 567 | 2,070 | 12,770 | 0.11% |
Total articles by publication year¶
| Latest data harvest | 13 November 2022 |
|---|---|
| First data harvest | 19 April 2022 |
| Total harvests | 30 |
| Data | https://github.com/wragge/trove-newspaper-totals/blob/master/data/total_articles_by_year.csv |
This chart shows the total number of digitised newspaper articles by year of publication. The distribution will change over time as digitisation priorities shift and more newspapers are added. This chart is useful in understanding how digitisation priorities and copyright restraints have shaped the total newspaper corpus.
Total articles by publication year and state¶
| Latest data harvest | 13 November 2022 |
|---|---|
| First data harvest | 19 April 2022 |
| Total harvests | 30 |
| Data | https://github.com/wragge/trove-newspaper-totals/blob/master/data/total_articles_by_year_and_state.csv |
This interactive chart helps you explore the distribution of articles by both publication year and state. Click on a state in the bar chart or legend to filter the results by year. Click anywhere in the background of the chart to reset. This enables you to compare digitisation patterns in different states and see how they contribute to the whole corpus.
Total articles added since 06 Nov 2022 by publication year and state
This chart shows the total number of digitised newspaper articles added since the previous harvest by year and place of publication. It's useful in seeing how search results from particular time periods and locations might be affected by current digitisation priorities.
Total articles added since 19 Apr 2022 by publication year and state
This chart shows the total number of digitised newspaper articles added since the first harvest by year and place of publication. It's useful in seeing how search results from particular time periods or locations might be affected by recent digitisation priorities.
Article categories¶
| Latest data harvest | 13 November 2022 |
|---|---|
| First data harvest | 20 April 2022 |
| Total harvests | 31 |
| Data | https://github.com/wragge/trove-newspaper-totals/blob/master/data/total_articles_by_category.csv |
| Current total | Change since 06 Nov 2022 | Change since 16 Oct 2022 | Change since 20 Apr 2022 | Percentage of total | |
|---|---|---|---|---|---|
| Advertising | 43,992,618 | 18,684 | 99,420 | 490,911 | 18.624% |
| Article | 163,908,157 | 27,539 | 150,531 | 1,115,697 | 69.390% |
| Arts And Culture | 3,991 | 13 | 70 | 621 | 0.002% |
| Classified Advertisements And Notices | 1,456 | 0 | 4 | 33 | 0.001% |
| Commerce And Business | 3,896 | 2 | 70 | 900 | 0.002% |
| Detailed Lists, Results, Guides | 26,373,364 | 2,889 | 17,484 | 121,403 | 11.165% |
| Display Advertisement | 465 | 0 | 0 | 1 | 0.000% |
| Editorial | 15,118 | 6 | 116 | 1,711 | 0.006% |
| Family Notices | 1,949,595 | 555 | 2,741 | 14,792 | 0.825% |
| Humour | 29,908 | 45 | 416 | 2,400 | 0.013% |
| Law, Courts, And Crime | 9,912 | 9 | 41 | 449 | 0.004% |
| Letters | 12,755 | 66 | 144 | 745 | 0.005% |
| Literature | 35,449 | 21 | 55 | 487 | 0.015% |
| News | 11,524 | 33 | 65 | 1,106 | 0.005% |
| Obituaries | 12,116 | 61 | 162 | 983 | 0.005% |
| Official Appointments And Notices | 1,094 | 0 | 1 | 29 | 0.000% |
| Puzzles | 44,561 | 50 | 303 | 4,629 | 0.019% |
| Reviews | 2,185 | 18 | 27 | 204 | 0.001% |
| Shipping Notices | 2,492 | 0 | 4 | 687 | 0.001% |
| Sport And Games | 16,919 | 90 | 212 | 2,379 | 0.007% |
| Weather | 11,757 | 23 | 184 | 2,400 | 0.005% |
Newspaper titles¶
| Latest data harvest | 13 November 2022 |
|---|---|
| First data harvest | 20 April 2022 |
| Total harvests | 30 |
| Data | https://github.com/wragge/trove-newspaper-totals/blob/master/data/total_articles_by_newspaper.csv |
As of 13 November 2022 there are 1,730 different newspapers in Trove.
| Total newspapers | 1,730 |
|---|---|
| Earliest publication date | 05 March 1803 |
| Latest publication date | 09 December 2021 |
Newspapers by state of publication¶
This chart shows the place of publication of newspaper titles in Trove.
Changes in the number of newspapers by state of publication¶
| Current total | Change since 06 Nov 2022 | Change since 16 Oct 2022 | Change since 20 Apr 2022 | Percentage of total | |
|---|---|---|---|---|---|
| ACT | 8 | 0 | 0 | 0 | 0.45% |
| International | 13 | 0 | 0 | 0 | 0.74% |
| National | 9 | 0 | 1 | 4 | 0.51% |
| New South Wales | 527 | 0 | 2 | 12 | 29.88% |
| Northern Territory | 8 | 0 | 0 | 0 | 0.45% |
| Queensland | 106 | 0 | 0 | 2 | 6.01% |
| South Australia | 154 | 0 | 0 | 0 | 8.73% |
| Tasmania | 115 | 0 | 0 | 0 | 6.52% |
| Victoria | 411 | 1 | 1 | 4 | 23.30% |
| Western Australia | 379 | 0 | 0 | 29 | 21.49% |
Newspapers added since 06 Nov 2022
| title_id | title | state | total | link |
|---|---|---|---|---|
| 1820 | Live Wire (Yallourn, Vic. : 1935 - 1955) | Victoria | 9667.000000 | https://nla.gov.au/nla.news-title1820 |
Newspapers added since 16 Oct 2022
| title_id | title | state | total | link |
|---|---|---|---|---|
| 1803 | Army Magazine (National : 1989 - 2002) | National | 2796.000000 | https://nla.gov.au/nla.news-title1803 |
| 1820 | Live Wire (Yallourn, Vic. : 1935 - 1955) | Victoria | 9667.000000 | https://nla.gov.au/nla.news-title1820 |
Changes in the number of articles from newspapers since 06 Nov 2022
| title_id | title | change in articles | link |
|---|---|---|---|
| 43 | Camperdown Chronicle (Vic. : 1875 - 1954) | 2,814 | https://nla.gov.au/nla.news-title43 |
| 250 | The Lilydale Express (Vic. : 1886 - 1897; 1914 - 1955) | 16,658 | https://nla.gov.au/nla.news-title250 |
| 1802 | Army (National : 1980 - 2021) | 1,519 | https://nla.gov.au/nla.news-title1802 |
| 1804 | Royal Australian Navy News (National : 1958 - 2021) | 18,886 | https://nla.gov.au/nla.news-title1804 |
Changes in the number of articles from newspapers since 16 Oct 2022
| title_id | title | change in articles | link |
|---|---|---|---|
| 43 | Camperdown Chronicle (Vic. : 1875 - 1954) | 2,814 | https://nla.gov.au/nla.news-title43 |
| 250 | The Lilydale Express (Vic. : 1886 - 1897; 1914 - 1955) | 16,658 | https://nla.gov.au/nla.news-title250 |
| 370 | Port Lincoln Times (SA : 1927 - 1965; 1992 - 2002) | 96,750 | https://nla.gov.au/nla.news-title370 |
| 1672 | Air Force News (National : 1997 - 2020) | 1,980 | https://nla.gov.au/nla.news-title1672 |
| 1802 | Army (National : 1980 - 2021) | 38,755 | https://nla.gov.au/nla.news-title1802 |
| 1804 | Royal Australian Navy News (National : 1958 - 2021) | 87,364 | https://nla.gov.au/nla.news-title1804 |
Total articles by publication state¶
Data about the number of articles by publication place (state) is available both from the main result API using the state facet, and from the newspaper API. As noted under Significant events, these totals don't always agree. I've included both below, so you can compare.
Number of articles by state of publication (via result API)¶
Number of articles by state of publication (via newspaper API)¶
Changes in the number of articles by state of publication (via result API)¶
| Current total | Change since 06 Nov 2022 | Change since 16 Oct 2022 | Change since 19 Apr 2022 | Percentage of total | |
|---|---|---|---|---|---|
| ACT | 3,296,997 | 0 | 0 | 0 | 1.396% |
| International | 854,022 | 0 | 0 | 2,842 | 0.362% |
| National | 657,328 | 20,405 | 124,406 | 204,584 | 0.278% |
| New South Wales | 82,163,804 | 0 | 12,929 | -735,869 | 34.784% |
| Northern Territory | 409,433 | 0 | 0 | 0 | 0.173% |
| Queensland | 41,128,577 | 0 | 0 | 38,641 | 17.412% |
| South Australia | 24,810,497 | 0 | 96,750 | 247,642 | 10.503% |
| Tasmania | 15,850,246 | 0 | 0 | 74 | 6.710% |
| Victoria | 41,789,081 | 29,139 | 29,139 | 1,526,186 | 17.691% |
| Western Australia | 25,247,083 | 0 | 0 | 557,281 | 10.688% |
Changes in the number of articles by state of publication (via newspaper API)¶
| Current total | Change since 06 Nov 2022 | Change since 16 Oct 2022 | Change since 19 Apr 2022 | Percentage of total | |
|---|---|---|---|---|---|
| ACT | 3,296,997 | 0 | 0 | 0 | 1.396% |
| International | 854,022 | 0 | 0 | 0 | 0.362% |
| National | 663,998 | 20,405 | 130,895 | 211,839 | 0.281% |
| New South Wales | 82,163,804 | 0 | 12,929 | 230,147 | 34.784% |
| Northern Territory | 409,433 | 0 | 0 | 0 | 0.173% |
| Queensland | 41,128,577 | 0 | 0 | 38,641 | 17.412% |
| South Australia | 24,810,497 | 0 | 96,750 | 249,718 | 10.503% |
| Tasmania | 15,850,246 | 0 | 0 | 74 | 6.710% |
| Victoria | 41,789,081 | 29,139 | 29,139 | 459,823 | 17.691% |
| Western Australia | 25,247,083 | 0 | 0 | 957,841 | 10.688% |
Significant events¶
While the changes documented here generally reflect the addition of new articles and newspapers, the total numbers can also be affected by system configuration and optimisation events, such as rebuilding the index. It's difficult to highlight these sorts of anomalies in the charts above, but where I notice them I'll add a note here.
31 May 2022 – reindex?¶
Between the harvests on 29 May and 5 June 2022 there was a dramatic change in the distribution of articles by state (using the state facet). The number of articles from NSW decreased by 939,065, and the number of articles from Victoria increased by 961,387. There were also some smaller changes affecting other states. There was a scheduled maintenance window on 31 May, which makes me think that these reflect changes in the underlying search index, rather than the actual number of articles. Further evidence for this is the fact that the number of articles per state calculated by using the newspaper API endpoint were out-of-sync with the article totals until the end of May. Since then, both sets of article totals have remained in sync. So, in short, the number of articles returned by the state facet between 19 April 2022 (when my harvests started) and 31 May 2022 can't be trusted.
Created by Tim Sherratt for the GLAM Workbench. Support this project by becoming a GitHub sponsor.