False Detection of Updates to PowerShell

[dking02]

I have been regularly receiving notification from UCheck 5.0.4.0 and 5.0.5.0-BETA of an update to PowerShell to version 7.4.101.0 or earlier versions of 7.4 preview (beta) channel. I have the latest version of the stable channel of PowerShell, version 7.3.9, 2023-10-26, installed on my PC. The "7.4.101.0" version number of the reported update appears to be an incorrect variation of the version number 7.4.0.101 for the 7.4.0-rc.1 preview (beta) channel release of 2023-10-24.

The latest releases of PowerShell are:
LTS (7.2): v7.2.16.500, 2023-10-26
Stable (7.3): v7.3.9.500, 2023-10-26
Preview (7.4): 7.4.0.101 (aka 7.4.0-rc.1), 2023-10-24   

References:
https://github.com/PowerShell/PowerShell
https://github.com/PowerShell/PowerShell/releases

The corresponding usage data for PowerShell (211 total users) that I collected from SUMo on the evening of 2023-10-31, just before it was taken offline, was:

7.4.0.101,beta, 5 users (2%)
7.4.0.6, beta, 26 users (12%)
7.4.0.5, beta, 1 user (0%)
7.3.9.500, current, 141 users (67%)
7.3.8.500, 33 users (16%)
7.3.4.500, 1 user (0%)
7.2.15.500, 3 users (1%)
7.2.1.500, 1 user (0%)

This data clearly shows that the majority of SUMo users were on the latest version of the stable channel, 7.3.9.500, not on latest or earlier preview or LTS channel releases. Please provide similar current usage data for PowerShell from the UCheck database for comparison. It apppears that UCheck has a fixation with erroneously reporting beta versions as updates.

While a beta/preview version may show some initial uptake of users when it is first released (as beta tester move to it), I would not expect the majority of users to migrate from the current stable release to the beta version over subsequent days. However, if the new version was actually for a stable release, most users would move to it upon receiving an update notification. This migration or lack thereof provides a possible way to distinguish between updates to beta and stable channel versions without knowledge of the underlying version numbering scheme and to automatically correct (retract) a false detection (FD) of an update after a day or two of data.

[Curson]

Hi dking02,

Thanks for your feedback.
This is indeed a problem.

The distribution we have is quite different.

5.7.0.18831 - 1800 Hits
7.4.101.0 - 423 Hits
7.4.0.101 - 335 Hits

In fact, the stable version is not present at all.
I don't know how SUMo managed to make the distinction between beta and stable version, but I would say it's time for us to find a reliable way to do so.

Regards.

[dking02]

I have not seen any follow-up on this problem.

The basic issue is that I (and most users) would be expected to be on the current stable version after a few days of its release. However, although I was on the latest stable release 7.3.9.500 at the time of my original post on 2023-11-14, it is not a even listed in the UCheck database data as you note in your response. Why not?

The 7.4.0.101 (and the incorrectly numbered 7.4.101.0) versions were (and still are) the latest preview channel version, 7.4.0-rc.1, 2023-10-24. From the data you provided, it appears that the actual versions reported by users are not being recorded in the database and the database is incrementing counters for later versions that will be reported as updates. These later versions are likely to be betas.  You need to be tracking the actual distribution of versions of users, not just trying to detect the "latest" version number above a small threshold. This might explain why there are a large number of false positives/false detects (FP/FD) of updates for community programs. This problem does not affect detection of updates to compatible programs.

I have suggested to Tigzy on two occasions to provide a capability for users to view the entries in the database of a program so they can detect what is going on. He has declined saying that would be a niche feature. SUMo had this feature and it was very useful into figuring out FD/FP of beta versions and observing the migration of users to new releases. SUMo had several moderators who were empowered to manually update the database to correct for FP attributed to beta versions.

Please note since my original message, the stable channel is now moved from the 7.3 to the new 7.4 release, version number 7.4.0.500, 2023-11-16. The LTS channel has also moved to 7.2.17,500, 2023-11-16. However, the preview channel is stuck at the old 7.4.0-rc.1 version (2023-10-24), since no subsequent preview releases have been produced with the release of 7.4.0. I am now on the stable channel version 7.4.0.500 but continue to get updates notifications to 7.4.101.0, which was the release candidate (7.4.0-rc.1) for that version.

[Curson]

Hi dking02,

You need to be tracking the actual distribution of versions of users, not just trying to detect the "latest" version number above a small threshold.

Do you know if that was the method SUMo used to detect beta versions ?

SUMo had several moderators who were empowered to manually update the database to correct for FP attributed to beta versions.

You mean that they were able to override SUMo's detections ? What happened when a new stable release was available ?

As a short fix, we will change PowerShell to a supported update.

Regards.

[dking02]

As you can see from the data for PowerShell 7 from the SUMo DB in my original post, there are flags to denote beta versions and a single flag to denote the current (latest) version of PowerShell 7. The algorithm to detect an update is automated but the flags can be manually changed by a moderator or developer if the detected update is reported and determined to be a false positive.

I do not know what algorithms were implemented in SUMo.  KC Software has declined multiple recent requests on the SUMo Forum to make the SUMo source code or algorithms available now that SUMo has gone EOL. However, if only a small percentage of users migrate to a new version after a day or two and the majority remain on a recent but earlier version, that new version is probably a beta.  if you do not track the actual versions that users are on, you are unable to see this migration pattern. Please note that PowerShell was cited here as an example of the problem. I think this same problem applies to false positives on other community programs.

Ref. the following posts on the Wilder's UCheck thread:

OLLI_S (#365, 2023-11-06, page 15): [SUMo offers a cool feature to report beta versions:
Right click the entry, select "Report Beta", here a new email is generated with all required data already filled in.

Tigzy (#368, 371, 2023-11-06, page 15): Yes but this goes against the full automation... Someone has to pick up the support ticket and do it. Not sure what load this would generate for us. . . I believe this is what happened with SUMo if an email was generated.

Bill H (#380, 2023-11-06, page 16) For SUMo, Kyle had members of the community that had the power to mark a version as a beta or unmark it as beta. I am one of the individuals that could do this. I'm not sure how many folks he had with this power. It wasn't all done by Kyle and his team. Link: Post #380

[Curson]

Hi dking02,

Thanks for the numerous data.

The ways SUMo did keep tracks of version numbers seems way robust than what we use now.
However, it looks like Tigzy was against the idea and, to be honest, I doubt I can change his mind at the moment.

Maybe making an entry on and collect support for the Community would be a better way ?

Regards.