VirusChat: COVID-19 Statistics

Discussion in 'COVID-19' started by Simon Hampel, 11th Aug, 2020.

Join Australia's most dynamic and respected property investment community
  1. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    With the weekly spreadsheet I've been updating for the past few months being such a manual process, I started to automate the collection of some of the data. This eventually lead me to creating a web application where I could visualise the data in ways that other websites wouldn't let me.

    It's still a work in progress - but here is the first release of the VirusChat :: COVID-19 Statistics website.

    (No, I'm not planning on starting a new forum for VirusChat - I just happened to have the domain viruschat.com.au sitting around unused :p )

    So far I have a table showing various statistics by country, plus tables for Australia, US and the UK showing data by state (or country/territory in the case of the UK).

    upload_2020-8-11_14-8-54.png


    upload_2020-8-11_14-9-34.png


    ... the columns are sortable - click on the heading to sort by that column.

    I do plan on adding more statistics and charts for comparison purposes.

    Data collection is automated and pulled directly from the JHU COVID-19 data which is updated at least daily.

    There may be some delay in seeing current data compared to what we see published locally - I have to wait for JHU to publish their updates which typically happens around 3pm Sydney time each day.
     
    Gockie likes this.
  2. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    I'll summarise what each column in the tables means (most of this info is summarised at the bottom of each page):

    • Cases: the number of confirmed cases as reported to JHU
    • Average daily cases: a 7 day moving average of new cases (each day, we average the number of new cases from the past 7 days)
    • Cases growth level: see post below
    • R number: compare today's moving average of new cases with yesterday's moving average of new cases. If R number > 0, then average new cases is growing - shows potentially exponential growth. If R number < 0, then average new cases is shrinking - shows spread is being contained. We only show R number if there are at least 100 new cases being found each day on average.
    • Infection %: the percentage of the population that has been infected
    • Deaths: the number of deaths as reported to JHU
    • Average daily deaths: a 7 day moving average of new deaths reported each day
    • Death growth level: see post below
    • Deaths per mil pop: the number of deaths per million head of population
    • Deaths per case: the number of deaths divided by the number of cases
     
    geoffw likes this.
  3. Lizzie

    Lizzie Well-Known Member

    Joined:
    9th Jul, 2015
    Posts:
    9,625
    Location:
    Planet A
    They are pretty sad stats ... :(
     
  4. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    Case growth level:

    This is a new measure that I've invented to give an indication of the relative growth in daily new cases on a week-on-week basis.

    This is an exponential indicator, so +2 is 10 times higher than +1 and +3 is 10 times higher than +2, etc.

    Here's how it is calculated:

    1. we start by calculating the number of new cases each day
    2. calculate a 7-day moving average of those daily new cases
    3. we compare today's average new cases with the average new cases from 1 week ago
    4. if today's average is higher (new cases is growing) then the number is positive, if it is lower, the number is negative
    5. we calculate the growth level using a factor of 10 based on the following table:
    • +4 Tens of thousands (+10,000 to +99,999)
    • +3 Thousands (+1,000 to +9,999)
    • +2 Hundreds (+100 to +999)
    • +1 Tens (+10 to +99)
    • 0 Less than 10 (-9 to +9)
    • -1 Tens (-10 to -99)
    • -2 Hundreds (-100 to -999)
    • -3 Thousands (-1,000 to -9,999)
    • -4 Tens of thousands (-10,000 to -99,999)
    When looking at the figures, if a country has a +3 growth level, you should read this as "that country has seen thousands of new cases added in the past week".

    Hint: the number represents x in the power 10^x ... so 10^1 = 10, 10^2 = 10x10 = 100, 10^3 = 10x10x10 = 1,000, etc

    If a country has a +4 growth level, you should read this as "that country has seen tens-of-thousands of new cases added in the past week"

    A growth level of 0 means "almost zero growth, could be positive, could be negative - but not much change"

    Right now Australia has a -1 growth level, which means that this week's average daily new cases is "tens" less than it was at this time last week. It's shrinking - but not very fast.

    No country has yet reached +5 growth level (100,000+ new cases in a day)
     
  5. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    Here's a stat I wasn't previously showing in my weekly updates - ranking of average daily cases - much more meaningful than simply comparing total number of cases.

    upload_2020-8-11_14-40-49.png

    Australia is currently ranked #47 for average daily cases - out of 82 countries with more than 10,000 cases.


    upload_2020-8-11_14-41-51.png


    upload_2020-8-11_14-42-32.png


    ... I'll set up a nicer looking report I can pull weekly to show how this is changing.
     
    Casteller and geoffw like this.
  6. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    I've released v1.1 of the VirusChat website.

    Fixes:
    • some tidy up of the page layout
    • fixed some sorting issues with one of the table columns (Growth level)
    New features:
    • Reports - simpler versions of each table focused around only the important data for each column
    upload_2020-8-13_13-22-17.png



    upload_2020-8-13_13-23-48.png
     
    Last edited: 14th Aug, 2020
  7. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    v2.0.0 has just been pushed live

    Fixes:

    Changes:
    • change date display to d MMM YYYY format to make it more universally understood
    • major rewrite of underlying path structure for consistency and extensibility
    New features:
    • added tables showing historic data

    We can now see historical data for each country and for selected states.

    upload_2020-8-14_9-29-44.png


    upload_2020-8-14_9-30-13.png
     
    Last edited: 14th Aug, 2020
    geoffw likes this.
  8. geoffw

    geoffw Moderator Staff Member

    Joined:
    15th Jun, 2015
    Posts:
    11,676
    Location:
    Newcastle
    A small suggestion - could you put in a 404 handler? You've changed some URLs - when I went there with my URL from yesterday, I got an unfriendly "not found".

    It's good that you've included explanations of statistics, like growth levels, at the bottom of pages.

    Could you include reverse sort order on column headings without too much work? If you've already clicked on a heading, clicking on it again would give reverse sort order, as Worldometers does.
     
  9. Mark F

    Mark F Well-Known Member

    Joined:
    29th Jan, 2020
    Posts:
    1,033
    Location:
    Canberra
    Interesting that the UK has reduced their covid deaths by about 5,000 but seem incapable of explaining the massive excess deaths figure.
     
  10. Lizzie

    Lizzie Well-Known Member

    Joined:
    9th Jul, 2015
    Posts:
    9,625
    Location:
    Planet A
    The removed from the "death from" list anyone who had tested positive from Covid - but didn't die within 28 days of being tested. Not sure how that works except to manipulate the figures.

    Apparently the rate on death certificates noted as being "from Covid" is around 10,000 more than official death figures.



    It's a bit like countries - such as the US - claiming their cases are falling, but is it because they're not testing as much?
     
  11. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    v2.1.1 pushed live

    Changes:
    • lots of tweaks to layout and some URL changes (with redirects so old links don't break)
    • today's cases and today's deaths added to reports
    New features:
    • date-specific reports and comparisons - go back in time to see what the status was as of any date
    • WoW rankings for some of the reports, showing how ranking has changed since the prior week - now using red for increase and green for decrease in ranking

    upload_2020-8-16_21-53-58.png
     
    geoffw likes this.
  12. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    Note that there seems to be an issue with the source data at the moment - JHU seems to have accidentally pushed yesterday's data (15th of August) over the top of Friday's data (14th August), so current figures might be unreliable. In theory the figures should be correct, but showing the wrong date - but I'm not 100% sure if that's what's actually happened.

    The VirusChat system is designed to automatically fix corrections published to the source data up to a week later - so hopefully it will all come good in the next 24-48 hours once they've cleaned up the data.
     
  13. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    Data from the 14th / 15th has been fixed by JHU
     
  14. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    v2.2.0 has been pushed live

    New feature:
    • List of top 10 countries and Australian states on home page
    • Sparkline charts showing average daily cases for countries / states listed on home page


    upload_2020-8-18_10-44-13.png


    upload_2020-8-18_10-44-27.png
     
    Casteller, wylie and geoffw like this.
  15. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    v2.3.0

    New features:
    • Home page sparkline charts now show full history of average daily case numbers to give more context. Charts are also larger.

    upload_2020-8-19_9-28-32.png
     
    The Y-man likes this.
  16. Gockie

    Gockie Life is good ☺️ Premium Member

    Joined:
    18th Jun, 2015
    Posts:
    14,781
    Location:
    Sydney
    Hi @Simon Hampel,
    Is this 3 July cases number for NSW correct? 903C6B0C-0A20-4C35-8144-0BB377EDE54F.png
     
  17. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    v2.4.0

    New features:
    • New country index with links to every country tracked by JHU
    • Zoom-able charts!

    upload_2020-8-23_22-0-51.png


    upload_2020-8-23_22-2-3.png
     
  18. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    That will be an adjustment in the JHU data.

    I've found that the state data isn't that accurate for Australia on a daily basis. Sometimes it can take a week or so for the data to catch up.

    They don't publish daily case counts - they only publish an overall total case/death count each day. I then calculate the difference from one day to the next to determine the daily count. The problem is if the data hasn't been updated - you'll get zero new cases for one day, then double the new cases for the next.

    The good thing about their approach is that the overall numbers are self-correcting, meaning that if you simply look at the most recent data, you'll get a pretty accurate indication of what the situation is. Comparing daily data is less accurate due to issues with the timing of updates into the JHU system.

    You'll find similar issues with data from some smaller countries too - if it's not updated daily you get very lumpy charts where there may be zero cases recorded for a few days and then suddenly a heap of new cases added. The moving averages help smooth this out somewhat - but it's not perfect.

    The country level data for the larger countries is generally more consistent - and the US country and state data seems to be regularly updated.
     
    Gockie likes this.
  19. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    @Gockie I was just looking at the NSW Health website and they have the explanation:

    "On 03 July, 189 historic cases reported in crew members on board a ship wee classified as Australian cases and included in NSW totals"

    So the chart is accurate - it is an adjustment figure.
     
    Gockie likes this.
  20. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,412
    Location:
    Sydney
    It was pointed out to me by someone from Johns Hopkins University that they have a more reliable source of data that what I was using - so I've made an edit to the VirusChat application and rebuilt the data to use the new source.

    If you visited the site today and were wondering why it wasn't working - that would be the reason.

    This has fixed a few anomalies such as the recent one for Colombia.
     

Buy Property Interstate WITHOUT Dropping $15k On Buyers Agents Each Time! Helping People Achieve PASSIVE INCOME Using Our Unique Data-Driven System, So You Can Confidently Buy Top 5% Growth & Cashflow Property, Anywhere In Australia