Data collection viewer

Microsoft have announced a new "data collection
viewer".

https://yro.slashdot.org/story/18/01...lects#comments

From the screenshot it's clear that the information
provided is virtually undecipherable to nearly everyone:

http://www.thurrott.com/microsoft/15...osoft-collects

Not suprisingly, Thurrott has chosen to screenshot
hardware info rather than things like a list of music
you've bought.

Interesting, Microsoft are presenting this new PR
as a move to increase trust and respect privacy!

https://blogs.windows.com/windowsexp...EMR8GFDzRDZ.97

It's also a very clever move to normalize their spyware.
Not only have you accepted their spyware by using
Win10. You and Microsoft are actually friendly partners
in respecting your privacy. You can't say you didn't
know what you were agreeing to. It's right there in
black and white:

cV: "zNWez09CsEmjsb5B.0"
epoch: "1102666"

Sort of like the crook who provides an invoice of all
items taken as he walks out with your sofa. Except
that the crook's invoice would no doubt be readable.

Mayayana wrote:

[snip]
Interesting, Microsoft are presenting this new PR
as a move to increase trust and respect privacy!
[snip]

I've been thinking about adding a firewall rule to block all outbound
connections to anything MS-related.

--
When she died, I should've cried and spared myself some pain.

Mayayana wrote:

Microsoft have announced a new "data collection
viewer".

https://yro.slashdot.org/story/18/01...lects#comments

From the screenshot it's clear that the information
provided is virtually undecipherable to nearly everyone:

http://www.thurrott.com/microsoft/15...osoft-collects

Not suprisingly, Thurrott has chosen to screenshot
hardware info rather than things like a list of music
you've bought.

Interesting, Microsoft are presenting this new PR
as a move to increase trust and respect privacy!

https://blogs.windows.com/windowsexp...EMR8GFDzRDZ.97

It's also a very clever move to normalize their spyware.
Not only have you accepted their spyware by using
Win10. You and Microsoft are actually friendly partners
in respecting your privacy. You can't say you didn't
know what you were agreeing to. It's right there in
black and white:

cV: "zNWez09CsEmjsb5B.0"
epoch: "1102666"

Sort of like the crook who provides an invoice of all
items taken as he walks out with your sofa. Except
that the crook's invoice would no doubt be readable.

Did you actually look at the screenshot of the XML form data? That
there are unique keywords to identify you should be no surprise to you.
Just look farther down the XML data to see what information is being
conveyed. That Microsoft uses bitmasked or solitary values to identify
a particular item or setting under a class should also be no surprise to
you. You've been in the registry before and know that a setting may
have numerical values which do not represent magnitude but instead a
particular selection.

The XML data provides the same information to Microsoft as to you except
you don't have their lookup reference to determine what the keywords and
values might mean. Well, as with long-time register users, you'll
eventually find out what those mean. Microsoft also doesn't document to
you what a numerical value means for a data item in the registry that
represent a selection rather than a magnitude. That's something you
have to dig for ... as before.

Ever been looking around in the registry under the Enumeration key? If
you had, you would realize that vendor IDs are not written in text you
can read plus many of those settings recorded for a USB's presentation
data won't make sense to a casual reader. You have to go lookup that
information. An XML dump of the Enumeration key would also look as
cryptic as the output of this app.

Do you only install Microsoft software in the Microsoft OS? No. You
want something else or something more than Microsoft provides hence the
appearance of 3rd party software. If someone feels like taking on the
task (are you offering?), they could write a wordy interpreter for the
XML output of this app, much like how there are many GUIs written to
make easier the use of ffmpeg.

Microsoft provided the information, just like they provided system APIs.
Let someone else add the enhancement, just like others using the system
APIs (which are not written for the general user populace to understand)
to write something more. Do you directly use ffmeg from the command
line or do you use a GUI frontend to ffmpeg? Did the makers of ffmpeg
write all those GUI frontends or did someone else? If there really is a
need by a significant portion of Microsoft's customerbase (which
includes corporate customers, not just end users) that want a more
detailed or lookup tool for the XML data, someone else will have to step
up to the task. Regedit.exe isn't everyone's cup of tea, either.

Mayayana wrote:
Microsoft have announced a new "data collection
viewer".

https://yro.slashdot.org/story/18/01...lects#comments

From the screenshot it's clear that the information
provided is virtually undecipherable to nearly everyone:

http://www.thurrott.com/microsoft/15...osoft-collects

Not suprisingly, Thurrott has chosen to screenshot
hardware info rather than things like a list of music
you've bought.

Interesting, Microsoft are presenting this new PR
as a move to increase trust and respect privacy!

https://blogs.windows.com/windowsexp...EMR8GFDzRDZ.97

It's also a very clever move to normalize their spyware.
Not only have you accepted their spyware by using
Win10. You and Microsoft are actually friendly partners
in respecting your privacy. You can't say you didn't
know what you were agreeing to. It's right there in
black and white:

cV: "zNWez09CsEmjsb5B.0"
epoch: "1102666"

Sort of like the crook who provides an invoice of all
items taken as he walks out with your sofa. Except
that the crook's invoice would no doubt be readable.

I finally got mine working, under 17083, and
there's really no "crypto puzzles" for me.
I'm disappointed. I was hoping to find one of those
mystery ones and then look for anything to
correlate it against.

I'm going to have to figure out how the Census Hardware
ones are created, so I have some materials to play with.

The DDV is in the Windows Store, so it has to be
installed to be present.

*******

My Windows 10 Insider hasn't received an update for
a while. And I discovered all my Flighting setting
have been damaged. By what, I don't know. I had to
change a few of them before 17083.1000 was offered.

https://answers.microsoft.com/en-us/...4f7404f?auth=1

You can take a screen shot of your settings similar to
these, and then try changing a few until updates
come in again.

https://fud.community.services.suppo...7-c49762811fdf

HTH,
Paul

"Paul" wrote

| I finally got mine working, under 17083, and
| there's really no "crypto puzzles" for me.

Not for you. But for the average person there's
probably nothing readable. It would require a
translation system.

| I'm disappointed. I was hoping to find one of those
| mystery ones and then look for anything to
| correlate it against.
|

I wonder how it connects with full vs basic.
You have the full spyware disabled? And can
the viewer even show the full spyware data?
I imagine some people might be a bit shaken
up to see their movie list flying up the pipe.
(How does that accord with the federal law
against tracking one's movie viewing, I wonder?)
Similarly with the collection of Cortana
interactions. It's one thing to accept Cortana.
It's another to listen to an audio record of what
has recorded.)
According to their own page (the most clear
description I've yet seen) it's virtually a real
time spyware log when full data is enabled,
bordering on a keylogger:

https://docs.microsoft.com/en-us/win...stic-data-1703

Choice of default programs, log of searches
in documents, choice of music and movies,
caret location in documents (in other words,
tracking as you read a document), shopping
history, etc. It's a very long list.

The basic level collection is detailed not in plain
English but more as API docs:

https://docs.microsoft.com/en-us/win...nts-and-fields

It still collects software info, file info, device info,
logs of events like elevation, default software
preferences, etc. But much of what's collected,
and how, is not clear. For instance, there's this:

Microsoft.Windows.Appraiser.General.InventoryAppli cationFileAdd

It's described as an "event", but it's detailed as a collection
of all possible metadata about a file, such as one might find
in Explorer and in the Properties - Version tab. Why? How
often? Which files? It's never explained. The whole thing is
presented as data to "keep Windows up to date", but even
a casual reading makes it obvious this is marketing
research. (Collecting info about software preferences, for
instance. But even collecting info about software and
versions of other companies goes way beyond system
maintenance.)


| I'm going to have to figure out how the Census Hardware
| ones are created, so I have some materials to play with.
|
Maybe you can write Vanguard's translator.

| The DDV is in the Windows Store, so it has to be
| installed to be present.

Mayayana wrote:
"Paul" wrote

| I finally got mine working, under 17083, and
| there's really no "crypto puzzles" for me.

Not for you. But for the average person there's
probably nothing readable. It would require a
translation system.

| I'm disappointed. I was hoping to find one of those
| mystery ones and then look for anything to
| correlate it against.
|

I wonder how it connects with full vs basic.
You have the full spyware disabled? And can
the viewer even show the full spyware data?

My Insider (the one where I can use DDV) is set to Full.

There's no signs they're going to show us anything
incriminating. Like any URLs they're snaffling, are
not in the ddv.csv Export file. It would appear they're
trying to impress us with the "benign" data.

I got a hardware census to run - I plugged in a webcam,
in an attempt to trigger it, and got some entries. I can't
say with any certainty, whether one action had to do anything
with the result.

You can press F5 in the DDV, to get the latest state
info displayed.

And the DDV data cache, there's a new entry in cleanmgr
to erase it when you want to.

So far the search engines are useless on the topic.
I might as well be shopping for lead overshoes.

Paul

"Paul" wrote

| There's no signs they're going to show us anything
| incriminating. Like any URLs they're snaffling, are
| not in the ddv.csv Export file. It would appear they're
| trying to impress us with the "benign" data.
|
| I got a hardware census to run - I plugged in a webcam,
| in an attempt to trigger it, and got some entries. I can't
| say with any certainty, whether one action had to do anything
| with the result.
|
So it's probably just the basic data that shows?
In that case I guess if you download Good Will Humping
there'll be no way of knowing whether Windows is tattling
to Satya.

| So far the search engines are useless on the topic.
| I might as well be shopping for lead overshoes.
|

I'm not surprised. There's an army of compliant
"windows insiders" like Thurrott who are very
good at doing Microsoft's PR bidding in the guise
of tough journalism.

I'm continually surprised, though, at the bland
response of almost everyone. The list of private info
sent back is stunning and potentially illegal. And
Microsoft have no claim to it whatsoever. Most
people using Win10 have not even bought Windows
from Microsoft.
It made me think of the Video Privacy Protection Act
that was enacted after Robert Bork lost his bid for
the Surpeme Court. His VHS rental history had been
leaked to the press and a law was passed to make
it illegal to share such history without reason. Yet
MS say they collect music and movie info. (They claim
to collect just about everything except titles, but that
data is clearly there. It includes URLs.)

Mayayana wrote:
"Paul" wrote

| There's no signs they're going to show us anything
| incriminating. Like any URLs they're snaffling, are
| not in the ddv.csv Export file. It would appear they're
| trying to impress us with the "benign" data.
|
| I got a hardware census to run - I plugged in a webcam,
| in an attempt to trigger it, and got some entries. I can't
| say with any certainty, whether one action had to do anything
| with the result.
|
So it's probably just the basic data that shows?

It could be.

https://github.com/Microsoft/Telemet...oid/PartA.java

if(correlationVector.isInitialized) {
tags.put("cV", cV);
}

We still don't know what a correlation vector is, but it
might have something to do with a "bucket" for storage
of the data. Perhaps "userid" times "application_name".

I suspect the two cV values represent a total 128 bit quantity
(like a GUID), but the representation is pretty strange. I mean,
who puts two fields with identical names in the same file ?
Is one the MS word, the other the LS Word ?

Nothing in these softwares I might run into, or any documentation,
need describe the content. It's a jolly secret. But at
least we have a name for it, something to add during
further searches. Maybe the cV is issued to an application
developer and they simply "stamp" their reports with it ?

The iKey may have something to do with PKI for transmission
of the data. The data will be protected by https say,
as one layer of protection, and Public Key Infrastructure
will provide another layer of protection. The idea being
that only Microsoft can decode the message when transmitted.

I don't know if there's any "danger" with sharing fields
like those with other people - screwing with them only
upsets the data integrity of the vortex server after all.
Or the integrity of reports some developer receives as
part of ApplicationInsight/CEIP or the like.

I guess I should type some racy URLs into MSEdge now,
and see if they show up in the diagnostic data viewer
application.

Paul

"Paul" wrote

| I suspect the two cV values represent a total 128 bit quantity
| (like a GUID), but the representation is pretty strange. I mean,
| who puts two fields with identical names in the same file ?
| Is one the MS word, the other the LS Word ?
|

I thought that was odd, too. I tried converting the first from
base-64, just as a longshot.
If MS were *really* making this data available there
would be a "plain-English-ification" of it. I suspect researching
it will turn up a frustrating mixed bag, like researching Google's
search parameters -- lots of gossip, half-truths and
maddeningly convoluted details.

| I guess I should type some racy URLs into MSEdge now,
| and see if they show up in the diagnostic data viewer
| application.
|
Sure. Look for a value that looks like PrvRT, or maybe
pvRAT. That will obviously be the Perv Rating. Then you'll
just have to decipher the bit flags in what will probably
be a 64-bit integer.

On Sat, 27 Jan 2018 09:17:58 -0500, Paul wrote:

Mayayana wrote:
"Paul" wrote

| There's no signs they're going to show us anything
| incriminating. Like any URLs they're snaffling, are
| not in the ddv.csv Export file. It would appear they're
| trying to impress us with the "benign" data.
|
| I got a hardware census to run - I plugged in a webcam,
| in an attempt to trigger it, and got some entries. I can't
| say with any certainty, whether one action had to do anything
| with the result.
|
So it's probably just the basic data that shows?

It could be.

https://github.com/Microsoft/Telemet...oid/PartA.java

if(correlationVector.isInitialized) {
tags.put("cV", cV);
}

We still don't know what a correlation vector is, but it
might have something to do with a "bucket" for storage
of the data. Perhaps "userid" times "application_name".

We probably don't know specifically, and I think that's what you're
saying, but we do know generally what a correlation vector is.

In the early data gathering days of organizations like Google or
Microsoft, they would have been pulling in billions of individual data
elements. The challenge wasn't in sucking things up, it's in putting it
together, or correlating it, similar to putting together a jigsaw
puzzle. You learn element A about person 1 and element B about person 2,
then you learn element C that ties those two people together, so now you
know element A *and* element B (and of course element C) about this
person. Element C was a correlation vector. Similar to the jigsaw
puzzle, the more you know, the easier it becomes to correlate even more.

In the data gathering world, there seems to be two main differences
between then and now. Certainly, the tools for gathering data and the
tools for correlating it have gotten much better, but perhaps more
importantly, the utter general lack of people's concern gives them
license to do more, to go further, all in the name of providing a better
user experience, of course. ;-)

Fast forwarding to now, the Googles and Microsofts of the world now see
that they were entirely wrong back then. They initially worried that
people would rebel against the huge invasion of privacy that all of this
is, but they've since found that people very willingly give up their
privacy when they think they're getting something in return, or when you
can bury permission to collect user data on page 17 of a 20-page license
agreement written in 6-point font.

I suspect the two cV values represent a total 128 bit quantity
(like a GUID), but the representation is pretty strange. I mean,
who puts two fields with identical names in the same file ?
Is one the MS word, the other the LS Word ?

I only see a single cV value. The other cV reference is always in
quotes, indicating that it's a label. Or are you talking about something
else?

Char Jackson wrote:
On Sat, 27 Jan 2018 09:17:58 -0500, Paul wrote:


I suspect the two cV values represent a total 128 bit quantity
(like a GUID), but the representation is pretty strange. I mean,
who puts two fields with identical names in the same file ?
Is one the MS word, the other the LS Word ?

I only see a single cV value. The other cV reference is always in
quotes, indicating that it's a label. Or are you talking about something
else?


The sample photo here shows what I'm seeing.
Two cV lines, one above the other.

https://blogs.windows.com/windowsexp...a-privacy-day/

Sample photo of DDV interface.

https://winblogs.azureedge.net/win/2...0a3df9d752.jpg

In the exported CSV file, they look like this, for some
random encoded alphanumeric strings "x" and "y".

""cV"":""xxxxxxxxxxxxxxxx.0"",""cV"":""yyyyyyyyyyy yyyyy.0"",

The real ones, look like the ones in the pictu zNWez09CsEmjsb5B.0

I suppose I should collect all the ones I have so far, and
figure out the symbol set. "/" and "+" are allowed in the string.

And I do see repeats of cV pairs, so they aren't as random
as I initially thought they were. (It's pretty hard to spot
patterns in .csv files, using Notepad :-) ) Maybe if it had
been dumped in XML it would have been easier.

And I do see some rows in my .csv file, which have only one cV in them.

Maybe I should hunt down the cache these are stored in,
and see if the format there is easier to work with.

Paul

On Sat, 27 Jan 2018 11:09:51 -0500, Paul wrote:

Char Jackson wrote:
On Sat, 27 Jan 2018 09:17:58 -0500, Paul wrote:


I suspect the two cV values represent a total 128 bit quantity
(like a GUID), but the representation is pretty strange. I mean,
who puts two fields with identical names in the same file ?
Is one the MS word, the other the LS Word ?

I only see a single cV value. The other cV reference is always in
quotes, indicating that it's a label. Or are you talking about something
else?


The sample photo here shows what I'm seeing.
Two cV lines, one above the other.

https://blogs.windows.com/windowsexp...a-privacy-day/

Sample photo of DDV interface.

https://winblogs.azureedge.net/win/2...0a3df9d752.jpg

Oh, OK, I don't see any problem with that. In that case, cV appears to
be the object type, and the part in quotes is the unique identifier. You
could probably have a virtually unlimited number of cV's, if there was a
need for it.

In my day job, the equipment that I work with takes a similar approach
so it doesn't look odd to me.

On 27-1-2018 17:09, Paul wrote:
Char Jackson wrote:
On Sat, 27 Jan 2018 09:17:58 -0500, Paul wrote:


I suspect the two cV values represent a total 128 bit quantity
(like a GUID), but the representation is pretty strange. I mean,
who puts two fields with identical names in the same file ?
Is one the MS word, the other the LS Word ?

I only see a single cV value. The other cV reference is always in
quotes, indicating that it's a label. Or are you talking about something
else?


The sample photo here shows what I'm seeing.
Two cV lines, one above the other.

https://blogs.windows.com/windowsexp...a-privacy-day/

Sample photo of DDV interface.

https://winblogs.azureedge.net/win/2...0a3df9d752.jpg

In the exported CSV file, they look like this, for some
random encoded alphanumeric strings "x" and "y".

""cV"":""xxxxxxxxxxxxxxxx.0"",""cV"":""yyyyyyyyyyy yyyyy.0"",

The real ones, look like the ones in the pictu zNWez09CsEmjsb5B.0

I suppose I should collect all the ones I have so far, and
figure out the symbol set. "/" and "+" are allowed in the string.

And I do see repeats of cV pairs, so they aren't as random
as I initially thought they were. (It's pretty hard to spot
patterns in .csv files, using Notepad :-) ) Maybe if it had
been dumped in XML it would have been easier.

And I do see some rows in my .csv file, which have only one cV in them.

Maybe I should hunt down the cache these are stored in,
and see if the format there is easier to work with.

Paul

Try to use notepad++
It "knows the format of several file types,
and uses color to mark layout.
And is a nice replacement for NP.