Data Explorer (SEDE)

« first day (2099 days earlier) ← previous day next day → last day (9 days later) »

10:03 AM

This was marked as status-completed but some entries seem to be still broken: Tags column of Posts table changed delimiter format. Many historical queries broken. Special tag rendering also broken

Q: Tags column of Posts table changed delimiter format. Many historical queries broken. Special tag rendering also broken

To the best of my memory, and in reading my past queries- some of which I re-run on a regular basis- the format of the Tags column of the Posts table in SEDE used to be like <starball><is><so><heckin><awesome>. However, the format seems to have recently changed to |starball|is|so|heckin|awesome|....

I should probably post a separate bug report when I have a bit of time.

Here is an example: data.stackexchange.com/mathoverflow/query/1837950/…

The same post - but with all entries shown (SELECT * FROM...): data.stackexchange.com/mathoverflow/query/1837951?id=467162

Or maybe this falls under the "work in progress" mentioned in the answer?

A: Tags column of Posts table changed delimiter format. Many historical queries broken. Special tag rendering also broken

This is fixed; sorry for any inconvenience. What happened An inefficient scalar function that took a flattened list of tags and swapped it from one denormalized form (|tag1|tag2|) to another (<tag1><tag2>) was inadvertently skipped when running the most recent SEDE and data dump processes. What w...

> the data has been corrected across the network (Stack Overflow finished just before 06:30 UTC) for Posts. Work still in progress for PostHistory.

3 hours later…

rene

1:06 PM

@Martin it loos like all but PostHistoryTypeId = 9 (that is tag rollback) have been corrected: data.stackexchange.com/meta.stackexchange/query/1837958 and I doubt there is work in progress on that. Given the man in charge is active, lets give him a ping /cc @AaronBertrand

Aaron Bertrand

1:43 PM

Possibly always been broken? The code pre-dating me had: = CASE WHEN PostHistoryTypeId IN (3,6)

Don't know I'll get bandwidth to fix this today

Martin

2:11 PM

Thanks for looking into that! I saw that the answer was edited a few minutes ago.

> data has been corrected across the network (Stack Overflow finished just before 06:30 UTC) for Posts. Work still in progress for PostHistory, the remaining item being PostHistoryTypeId = 9. This should be corrected by the refresh on April 28th, 2024.

@AaronBertrand It is very likely that somebody would have noticed. (There must be many queries relying on that format.)

Looking at an xml-file I downloaded a few years ago, it seems to have the <tagname format.

For example, it contains this:

<row Id="17385" PostHistoryTypeId="9" PostId="7639" ... Text="&lt;reference-request&gt;&lt;nt.number-theory&gt;"

Aaron Bertrand

2:42 PM

@Martin Maybe? I'm new to the codebase, and I don't have all the history, and it's not the same that you'll find out in gists from a decade ago. I can tell you that I spot the problem and know how to fix it, but I'm not going to spend time trying to analyze who broke it and when.

I can imagine how it could be missed, since it isn't a very frequent action, relatively. data.stackexchange.com/stackoverflow/query/1837970

That's 0.018%, and that's just of posts that haven't been deleted and history that we expose in SEDE, since PostHistory doesn't have rows for deleted posts and not all types are exported. So, the % is even lower overall: 0.013%. I did provide a workaround for the query format that will support both types (and honestly it would be easier for everyone if that were used, since then we wouldn't have to do any transformations).

Just need to change LIKE '%<' + TagName + '>%' to LIKE '%[|<]' + TagName + '[|>]%' then you will be insulated from any further mess-ups (by me or anyone else).

Martin

3:10 PM

@AaronBertrand It's not very important, but I have personally hard time imagining that I would have missed that in the past.

There are several queries looking at the history of the tags that I regularly use. (And I am pretty sure some other users use such queries as well.)

Aaron Bertrand

@Martin I'm saying missed by whoever implemented any other changes to the code between a few years ago and the changes I've made over the past three weeks.

Martin

I am not talking about the code but about the people actually using SEDE.

How would this go unnoticed?

I am pretty sure that at least once in a month or two I run query which relies on this.

Yes, you're right that it is easy to change one line in the query. But going through all my queries using this would take quite some time...

Aaron Bertrand

Like I said, I don't have the code history, so I'm not sure what answer you're looking for aside from I am fixing it.

Martin

I am just responding to your conjectures such as "Possibly always been broken?" and that it might have been broken for a long time.

Of course, if it remains this way, I will learn to live with that - even if it might need going through several older queries and changing them. (But I'm hoping that I won't have to do that.)

Aaron Bertrand

I've already committed to fixing it for the next refresh, so no, you won't have to do that. I am speculating how it might have been missed, the code with (3,6) is code I came across, not code I made up. Again, not sure what more you want than that.

Martin

3:16 PM

Re: I'm not going to spend time trying to analyze who broke it and when. That's understandable - and I never said something like that was needed. All I said that something like this would get noticed.

Re: more you want than that. Nothing actually - I was trying to respond to you.

For example I have used this query many times in the past - on many various sites: data.stackexchange.com/mathoverflow/query/1229812/…