Track historical stats

[iandunn]

We'll need to track progress over time, to see if the program is being effective.

By default we'll be able to get a snapshot of the metrics at any point in time, just because the raw data will be in the database, but if we want to see historical trends (without which the data is pretty useless) then we'll also need some way to save snapshots at regular intervals, or store historical data, or something.

The data will be made available via #38

[iandunn]

Some potential metrics:

Misc

• number of total pledged hours across all companies/teams/etc
• number of people contributing (regardless of how many hours)
• number of companies contributing (regardless of how many hours)

Teams

• percentage of teams with at least 5 hours per week sponsored
• percentage of teams with at least 15 hours per week sponsored
• percentage of teams with at least 35 hours per week sponsored
• total # of pledged hours per team

Those are important b/c contributors with larger blocks of time can take on different types of projects, often ones that are difficult to complete otherwise, including boring/tedious administrative work.

[iandunn]

Maybe we just have a monthly cron job that gathers all of the above stats and saves them in a CPT or something?

[iandunn]

All of the individual stats are derived from data in the database, but tracking month-over-month growth will require saving snapshots of the data, and I think that's something we should do soon, so that we have data from when the project launches.

cc @coreymckrill

[coreymckrill]

So a snapshot might be:

• Number of pledges
• Number of contributors
• Total number of hours pledged

Anything else?

[iandunn]

🤔 I was thinking the snapshot would be of all of the stats that we track -- see #37 (comment) for a rough idea -- so that the public report would just load them without having to calculate anything.

Snapshotting the underlying data that the stats is derived from might be better, though, good point. I'll think on it some more, but I'd probably be fine w/ either.

[coreymckrill]

Re: percentage of teams with at least 5 hours per week sponsored etc.

If a contributor lists that they are involved in multiple teams, how would we determine what portion of their weekly hours go do each team?

[iandunn]

🤔 , good question. I guess we can't, unless we just assume they're split evenly? That wouldn't be 100% accurate, but maybe good enough for this purpose?

Or maybe, if we feel like that stat is important, then we should ask for a breakdown of hours on Profiles?

[iandunn]

@andreamiddleton, do you feel like #37 (comment) are the right stats to track? is there anything else that we should?

[iandunn]

number of people contributing (regardless of how many hours)

Should that include people who aren't sponsored by companies? If so, then we should probably include them in #27.

We'll also want to de-dupe the (rare) people who are sponsored by multiple companies, see #55 .

[iandunn]

It might be nice to also do a p2 post on make/updates or something once a month when the new stats are available, to help people keep an eye on progress. or maybe that'd be too noisy and people will just check the public report (#38) naturally?

[iandunn]

Not gonna get to this before my vacation, so un-assigning.

[iandunn]

Since we can't accurate track associate sponsored hours with individual teams, maybe we just track the # of contributors sponsored for each team.

I think that's a very poor metric, but it's better than nothing 🤷‍♂

[iandunn]

Rather than have a post for each month, and meta values for each stat, I think it might be better to have a post for each stat, and meta values for each month.

It seems like that'd avoid having to regroup everything on the fly every time the shortcode is rendered. Still playing around with it, though.

[iandunn]

Er, no, I think the original structure will work better. I've got a rough sketch of that working in the latest #81 commit (43b865c)

[iandunn]

Even after #81 is ready, we might want to wait to start tracking until after #83 is done, and all the existing data has been corrected. Otherwise it'll appear as if there was a large drop in participation, when really it was just inaccurate data being corrected.

[iandunn]

Preliminary stats:

[confirmed_hours] => 2316
[confirmed_team_contributors] => Array
    (
        [Core] => 76
        [Community] => 74
        [Meta] => 23
        [Docs] => 0
        [Polyglots] => 50
        [Security] => 17
        [CLI] => 19
        [Documentation] => 16
        [Support] => 28
        [Plugins] => 47
        [Design] => 14
        [Hosting] => 20
        [Marketing] => 11
        [Themes] => 21
        [TV] => 9
        [Tide] => 2
        [Test] => 5
        [Training] => 8
        [Accessibility] => 11
        [Mobile] => 3
    )

[confirmed_pledges] => 36
[confirmed_contributors] => 232

You see how Plugins is really inflated (only 3 out of 47 are accurate -- see #83), which I think it's safe to assume is impacting hours a lot as well.

[andreamiddleton]

@iandunn I think the stats that you called out are good ones to track, yes. :)