Tavern on the Meta

1:00 AM

111

Q: Stack Overflow for Teams is now free for up to 50 users, forever

Today we’re launching our Free plan on Stack Overflow for Teams. We know that sharing knowledge and async collaboration is critical to companies and teams of all sizes. We also know people need a good sized group and time to see the impact and value of a platform like Stack Overflow for Teams. Ou...

huh, this is neat

@JourneymanGeek If only SD cards were actually reliable.

Journeyman Geek

@TheforestofReinstateMonica they arn't

I think a good chunk of my side of the conversation, other than the observations that no storage should be trusted shouldn't be taken seriously ;)

The forest of Reinstate Monica

:p

Journeyman Geek

but the idea of a little mechanical SD card loader/unloader that goes WHIRRRRRRR ....

The forest of Reinstate Monica

I read some neat info about a technique to hack microSD cards and run arbitrary code on them, since they actually contain a fairly powerful processor inside (either ARM or heavily-modified 8051 core). I think that'd make for some very promising anti-forensics techniques.

But I would not be surprised if it was possible to hack such cards to improve their reliability.

By adding ECC code and the like (depending on failure mode, I guess).

Journeyman Geek

or just store multiple copies of it

The forest of Reinstate Monica

1:05 AM

Yeah but that wastes space.

ECC is more flexible.

Journeyman Geek

and it might be 'easier' to do that at a file system level

ECC dosen't deal with it at the "my whole potato SD card died"

The forest of Reinstate Monica

I don't know enough about SD card failure modes, sadly.

But I imagine a lot of it is simply the CPU freaking out because a single flash page dies.

Also, if they would actually use static wear leveling like they should, things would be a lot more rosy. But as it is, their FTL uses only dynamic wear leveling...

I wonder if it'd be useful to implement static wear leveling in software, above the filesystem. Check the mtime of a file and move it if the mtime is from too long ago, perhaps (or btime/crtime)?

Would that be a question to ask on SuperUser?

Journeyman Geek

uhh

Bit too high level for us I suspect

The forest of Reinstate Monica

hm

Journeyman Geek

(and heh, one of the blockers for a question I want to ask is pretty much "someone needs to write a program for that" )

The forest of Reinstate Monica

1:16 AM

Well it could also be done manually. E.g. "this 100 MiB file on my ancient CF card has been here for 5 years. I should cp file file2; mv file2 file so that 100 MiB of flash can be put to use".

Obviously true static wear leveling is much more involved and needs to be implemented in the FTL, but such a "ghetto solution" may or may not have a measurable effect on reliability.

@JourneymanGeek Is there no SE site where such a question would fit?

Journeyman Geek

@TheforestofReinstateMonica I really have no idea

The forest of Reinstate Monica

aw

9

Q: Area 51 uses insecure avatar generation

Area 51 uses a different system for avatars than the rest of the network. While on every other site Gravatar URIs are salted, on Area 51 they aren't because it hasn't been updated (Area 51 uses a really old fork of the SE engine). This could allows determining the email address associated with an...

bug area51 privacy security gravatar

lol nice

@JourneymanGeek Would publishing results from a script to analyze the "insecure" gravatars on Area 51 (in area51.stackexchange.com/users?tab=newest) be against Stack Exchange TOS?

Journeyman Geek

1:40 AM

@TheforestofReinstateMonica I think there's a process for proper disclosure

but I think SE staff being aware is enough, and they do read MSE these days

The forest of Reinstate Monica

(I wasn't the one who disclosed that issue)

Journeyman Geek

If its 'statistics' I guess it would be fine

if its actual email addresses, I would say hell no

The forest of Reinstate Monica

It's "user ID + IP address or email, reversed from e.g. 6149866d79f6d13ea188b10459de53ea.

Journeyman Geek

oh, hell no

The forest of Reinstate Monica

I'll keep it private then.

Of course anyone could publish that info even if I don't, since it's out there lol

You can tell security is bad when a rainbow table is still useful. :D

Journeyman Geek

1:45 AM

Oh, its a known/old issue

The forest of Reinstate Monica

For gravatar not using salted hashes?

Journeyman Geek

A51 is on a practically prehistoric codebase

@TheforestofReinstateMonica funny story that

The forest of Reinstate Monica

(I don't like the term "salted". I think it'd be better called "customized")

Oh I'm aware:

10

A: Is it possible to "reverse engineer" identicons?

It used to be possible, as Gravatar identicons are a graphical representation of the MD5 hash of an encoding (trim leading and trailing whitespace and convert letters to lowercase) of your email or IP address. Both of these things are small enough to make brute force feasible. Nowadays, Gravatar ...

Journeyman Geek

For some reason we've never really had a successful startups site

and at one point, someone basically reverse engineered the emails of every single user on one of the iterations of it and mailed them asking them to join a new, non SE Q&A site

The forest of Reinstate Monica

hah

That must've been before SE used salted identicons globally.

Journeyman Geek

1:47 AM

which if I recall why they started doing that

The forest of Reinstate Monica

Link to source (so I can include it in my sec.se answer)?

Journeyman Geek

uhhh

Its somewhere on meta but I don't remember enough details to find it

The forest of Reinstate Monica

ok

Journeyman Geek

.... I lie

10

A: What is Brightjourney?

I doesn't appear to be a scraper site, or really powered by Stack Exchange at all, just a sort-of knock-off inspired by a failed Area 51 site. From What happened to Startups SE?: Apparently it was resurrected as Bright Journey. I just got an email about this. - Kenny Evitt Which isn't quit...

... How the hell i remember something in the comments I'll never know

They did restore the Answers.Onstartups data dump back in the day, if I recall correctly. Which is sort of why we do data dumps, so that's fine, except that then they also emailed users, which was how we finally got rid of the gravatar hashes in data dumps. :/ (Disclaimer: this is true assuming my memory is holding up.) — Adam Lear ♦ Jun 23 '16 at 23:20

See also meta.stackexchange.com/questions/291029/…

but I have no idea if this is related

The forest of Reinstate Monica

thx

Journeyman Geek

1:56 AM

And

205

Q: Is Gravatar a privacy risk?

I ran across the comment below at http://onemansblog.com/2007/02/02/protect-your-privacy-delete-internet-usage-tracks/#comment-58200 about Gravatar. I'm particularly curious of Meta Stack Overflow's opinions on points 4 and 6, though the others may be of interest too. Are these concerns real, and...

discussion gravatar privacy

many of the answers here

and...

Yeah, in general if not for A51 kinda not being a very old codebase, it would be fixed

The forest of Reinstate Monica

I wonder what other security issues Area 51 has.

I recently (briefly) looked at MO to see if it was particularly buggy and it wasn't.

Journeyman Geek

MO is on the modern code base

The forest of Reinstate Monica

I thought it used the same old codebase.

Oh that explains it.

Journeyman Geek

A51 has a ... ton of customisations over the SE 2.0 codebase

(I'm not sure the current version has a name like that)

I'd have jokingly suggest replacing it with an array of teams instances

The forest of Reinstate Monica

lol

A51 is pretty ugly and difficult to navigate through, tbh.

Journeyman Geek

1:59 AM

Old codebase/design

most of SE was like that once

The forest of Reinstate Monica

@JourneymanGeek Wow, that brightjounrey thing is still up.

brightjourney.com - is it just a mirror of SE now?

Journeyman Geek

Well - quite a few 'replacements' start off with data dumps

The forest of Reinstate Monica

brightjourney.com/q/new/1

lolol

0 votes, 0 answers over and over.

The forest of Reinstate Monica

2:17 AM

@JourneymanGeek Re-read the comment. Seems that's about them blacklisting gravatar hashes in data dumps (to prevent exploitation of older hashes), not about using the salts.

Journeyman Geek

2:35 AM

I mean

they're aware its an issue

PM 2Ring

2:51 AM

@TheforestofReinstateMonica You can do that on the whole device using dd to copy /dev/sdc (or whatever) to itself. Use a block size of 64k so it doesn't take forever.

The forest of Reinstate Monica

3:22 AM

@PM2Ring Would have to do conv=notrunc though if you did it on individual files.

@PM2Ring But doing it on the whole device would defeat the purpose. The point of static wear leveling is to take blocks that have been sitting around for a long time and "free up" those flash blocks so they can go to work, and give some other "worn out" flash cells time to rest by giving them long-lived data. Otherwise long-lived data is stored in flash cells that still have 95% of their life left, and the short-lived data is stuck on degrading cells with a fraction of their writes remaining.

That's contrasted with dynamic wear leveling (done on SD cards, cheap USB flash drives, etc.) which simply randomize the physical location of data so two writes to the same logical sector don't hit the same flash blocks twice. Static wear leveling does both that, and periodically moves long-lived data to free up those healthy cells.

PM 2Ring

3:42 AM

@TheforestofReinstateMonica Well, it's a compromise. It's oblivious to the age of the data on the block, and just rewrites everything. But it's a lot faster than file by file re-writing. :) It's probably most useful on drives that get read often, but don't get a lot of writes. And you don't want to do it very often, maybe once every year or so.

The forest of Reinstate Monica

@PM2Ring Oh I think you misunderstood me. It wouldn't be helpful to rewrite all the files (that would probably do more harm than good in terms of flash wear), but it could be helpful to rewrite only the oldest, largest files, which would be a fraction of the total size of the device.

For example, once a year check for any files which are unchanged in the past five years and which are greater than 1 MiB in size and rewrite them (obviously the actual parameters would have to be chosen carefully to optimize this hacky static wear leveling).

@PM2Ring The glibc source code actually reports in the comments on some neat experiments they did to determine the optimal block size. IIRC it was something like 256k for the average device.

Which is why an strace of a program linked with glibc that does I/O will show 256k reads/writes.

read(7, "all work and no play makes jack a dull boy", 262144) = 42

PM 2Ring

@TheforestofReinstateMonica No, I get what you mean. And yes, my method does cause a bit of wear, so you definitely don't want to do it often. It's probably more useful on magnetic drives, to counteract the slow deterioration of the magnetic bit patterns.

The forest of Reinstate Monica

Ah, I was only talking about low-end flash devices with dynamic wear leveling.

@PM2Ring I don't think modern PRML/EPRML hard drives need "refreshing" like that.

And flash devices only need power every decade or so to retain data.

PM 2Ring

@TheforestofReinstateMonica Several years ago, I wrote a script to do dd speed tests. The optimum was from 64k to 512k, but there really wasn't a lot of difference in that range. But that was some years ago, on a slow 32 bit machine. And I mostly tested magnetic disks, not flash.

The forest of Reinstate Monica

The speed is not affected by the drive type, since the "block size" for the hardware is always 512 bytes due to the ATA protocol. A larger block size is a tradeoff between less context switching overhead and more memory accesses/TLB/cache overhead. The block size specified by dd and used by the I/O syscalls is lost when it hits the AHCI driver and the SATA/whatever hub translates it to 512 bytes.

@PM2Ring If I remember the glibc source's explanation correctly, they came to the same conclusion you did, that 64k-512k is optimal, but 256k was very slightly better on average (128k on some devices).

Syscalls like sendfile() of course make things a lot more efficient since there's no context switch overhead and no block size to specify. Of course, not everything uses sendfile() (I think busybox uses it for cp and friends?), and I'm not sure if it supports filesystems that use holes like ext4.

PM 2Ring

3:55 AM

@TheforestofReinstateMonica Sure. As you say, the timing difference is due to the CPU context switching & number of memory accesses.

The forest of Reinstate Monica

I think the best way to test it is with tmpfs (so you're only measuring VFS performance, not drive caching behavior or whatever) so you don't have to sort through all the variables that affect the underlying hardware. But filesystem buffer probably mitigates much of that.

PM 2Ring

And of course with a flash drive, it then translates those 512 bit blocks into its internal blocks.

The forest of Reinstate Monica

The FTL does crazy magic. Makes optimizing for performance a lot harder than with spinning rust.

My solution is just: MOAR RAM MOAR CACHE lol

Idea for the asshole hacker: 1) get root, 2) change the glibc default bs to 1, 3) ???, 4) profit!

Next up, on "'I symlinked /dev/zero to /dev/urandom' and other adventures of the asshole hacker"

PM 2Ring

I once had an idea of making a filesystem that implements an extra layer of EEC, kind of like a RAID system of cheap flash drives (of course all modern drives use some form of EEC internally). So using (7, 4) Hamming code with 7 16 GB drives you'd get 64 GB of storage that could recover from 1 bit errors. If any drive loses too many sectors, you just replace it.

The forest of Reinstate Monica

ZFS does that.

(If my memory of ZFS is correct. Dunno if OpenZFS does it yet)

brb kid needs food

PM 2Ring

4:06 AM

@TheforestofReinstateMonica No worries.

I originally thought of the (7,4) thing back when I was running an Amiga, with SCSI HDs: you can put 7 drives on a standard single SCSI bus. I wrote some Proof of Concept code in C, that just used 7 normal files to store the data. It was fast, and worked ok. But I never got around to learning the low-level stuff I'd need to implement an actual filesystem. I guess it wouldn't be too hard to do on Linux...

OTOH, I guess there's not much point if ZFS can do it. And proper SSDs are getting quite cheap, and very reliable. But I guess there'll always be a market for cheaper reliable storage, even if it is a bit slower.

Journeyman Geek

4:58 AM

just to point out, I wasn't being serious about the massive array of SD cards with a loader robot :D

The forest of Reinstate Monica

@PM2Ring A great resource for learning the basics in implementing a filesystem driver is the in-tree romfs filesystem for Linux, which is an ultra-lightweight read-only filesystem whose code entirely fits in one 4096 byte page of kernel memory. See kernel.org/doc/html/latest/filesystems/romfs.html

PM 2Ring

@TheforestofReinstateMonica Thanks!

The forest of Reinstate Monica

It's great for understanding how to interface with the VFS layer in Linux. Of course, the model filesystem that also supports writes is the FAT series. FAT12, FAT16, and FAT32 are basically the same, and there are a million drivers implementing it for a million different systems from Linux to NetBSD to TempleOS to FreeRTOS. But as simple as it is, romfs is even simpler (while still being functional).

Actually, I think FreeRTOS supports some really neat modified version of FAT which is entirely atomic by alternating between the first and second FAT, or something? I forget. (seems I was thinking of en.wikipedia.org/wiki/Transaction-Safe_FAT_File_System, which I seem to have remembered wrongly as a FreeRTOS thing?)

PM 2Ring

@JourneymanGeek Sure. But it's still a fun idea. And by using an error correcting code, you could get away with using really dodgy SDs (or other flash drives). You only lose data if 2 or more drives have errors on the exact same logical bits.

The forest of Reinstate Monica

RAID6 does that (albeit with the write penalty, since writing to one drive requires reading from the parity drives).

PM 2Ring

5:15 AM

And Hamming is just the simplest EEC. You can make codes that can recover from higher numbers of bits per word. And that can at least detect errors that are worse than what they can correct. IIRC, standard CD encoding can correct single bit errors & detect 2 bit errors.

The forest of Reinstate Monica

And I think DVDs ECC is even better.

Fun fact: The Twofish block cipher uses a Reed-Solomon ECC for cryptographic purposes (part of its linear layer).

PM 2Ring

With my (7, 4) system, each read or write requires accessing all 7 drives. You could put the 7 blocks onto a single drive, but it's more robust to put them on separate devices. And you can easily replace any 1 of the 7 & reconstruct it.

The forest of Reinstate Monica

Yeah same with most RAID5/6 setups.

Did your system share the same bus for all 7 drives?

PM 2Ring

@TheforestofReinstateMonica Yes.

The forest of Reinstate Monica

That'd certainly slow things down I bet.

PM 2Ring

5:26 AM

Just a little. ;) OTOH, you could do it with a "smart" hub, so to the OS it behaves just like a single drive, and the hub handles all the ECC stuff.

Journeyman Geek

@PM2Ring I used to work in a studio and we had a tape robot

The forest of Reinstate Monica

Did any extant hardware support that?

Journeyman Geek

so... an SD card robot... :D

@TheforestofReinstateMonica they exist

The forest of Reinstate Monica

@JourneymanGeek I meant @PM2Ring's "smart hub".

(I've seen tape robots before)

Journeyman Geek

basically an SD card raid to sata

PM 2Ring

5:32 AM

@TheforestofReinstateMonica No idea, but it doesn't have to be very smart. Hamming just requires a few simple bitwise operations. Modern hardware could easily do that quickly.

The forest of Reinstate Monica

sure

That's a common reason people like using hardware RAID controllers, but in reality, hardware RAID is often sub-par (though not as bad as firmware "fakeraid") because, well, the devs are idiots.

PM 2Ring

:)

The forest of Reinstate Monica

I was once stuck with hardware RAID so old that the drives would sometimes randomly disconnect and it took running crappy closed source obscure tools from an archive to fix the array.

The wires were frayed and some drives were just "hanging there".

PM 2Ring

Scary

The forest of Reinstate Monica

Yeah. I did not like it. Worst part is it was RAID0 with I think 6x decade old drives and we were too scared to try to change it to RAID6 for fear of breaking something. It was a miracle it survived.

Journeyman Geek

6:19 AM

speaking of which... I need to set up the first tier of server backups :D

The forest of Reinstate Monica

From SE's 404 page:

Isn't this image copyrighted or something?

(I'm not very familiar with copyright law)

VLAZ

7:23 AM

@TheforestofReinstateMonica I don't think it is. Copyright is for a work. If this was a photo an artist took, they'd have copyright over it but it's a still from a show. Usually those fall well under Fair Use.

If the 404 page played an entire episode of Star Trek, or at least a very significant portion of it, without needing to, then that might be a copyright breach.

The forest of Reinstate Monica

@VLAZ But it's a photo of a copyrighted work, placed on a for-profit website.

VLAZ

A photo of copyrighted work still isn't necessarily copyrighted.

The forest of Reinstate Monica

oh

VLAZ

Where it's placed doesn't matter that much. Fair Use allows for reviews and parodies, for example.

The forest of Reinstate Monica

Ah, I suppose this counts as a parody (the funny hats are not in the original show).

VLAZ

7:26 AM

Things you get on sites like reddit.

If any reproduction ever of any copyrighted work was banned, we wouldn't really have a lot of the internet.

The forest of Reinstate Monica

Oh I'm not trying to get it changed. I'm of the mind "pirate all the things".

Just curious if it was an oversight or intentional.

VLAZ

I'm just saying that if this wasn't allowed, it's far from the worst offender. A review website is for profit and often shows multiple stills and probably clips from movies/shows. With such reviews coming out weekly if not more often.

Journeyman Geek

in theory, if we couldn't use stills in fair use, quite a few of our media centric sites wouldn't be able to run too

or quote books (where the primary source is discussed)

or...

The forest of Reinstate Monica

Well RPG seems to have a fit if anyone posts "copyrighted text" from DnD :D

Journeyman Geek

I don't know the full context but I'm sure there's a story behind it

and sites are free to choose their own lines of what's ok, like our soon to be obsolete rule about hackintoshes :D

VLAZ

7:41 AM

@TheforestofReinstateMonica Really? I thought I've seen posted excerpts of rules there.

Journeyman Geek

I vaguely recall the issue was folks quoting sites that hosted large chunks of D&D books

The forest of Reinstate Monica

@VLAZ You sure it's from the official DnD source not some other game?

Journeyman Geek

Somewhat beyond fair use

VLAZ

The vast majority of my visits are from the HNQ so - yes.

I can't remember the last time I've seen a non-D&D post there

I don't tend to visit because....i.imgur.com/PIHbnwK.png

Look at how much of it is just D&D

And it's all questions that I really don't care about. "If my bard stands on their left foot, does that mean that X spell does Y instead" type of stuff.

Rules finagling.

Journeyman Geek

lol

Can't remember an RPG group that didn't either rules lawyer, or throw the rulebook out of the window ;)

VLAZ

7:49 AM

One of my most enjoyable RPG sessions was when we had 2-3 incomplete groups (we gathered to play multiple games) and so we just decided to roll up a game set in Discworld spontaneously. Well, one girl had an idea for a game and wanted to test it, so she just GM-ed.

But all we really had was a few dice.

Sonic the Curiouser Hedgehog

8:45 AM

@JNat Cross-site spammer: stackexchange.com/users/21016609/abhinandan-poll

rene

→ 10 messages moved to Chimney

Meta Andrew T.

9:07 AM

7

Q: Is there any reason why quoting a D&D 5e spell in its entirety would not qualify as "fair use"?

One of the Site Mods has an issue with this question I asked about a specific spell. They have stated that it is not right to quote the whole spell due to "fair use". I disagree and feel that keeping the spell as a whole is better for people who are researching the spell and might have question...

discussion moderation copyright

Rob

9:36 AM

I guess it was thought that mentioning if it was fair use would be fair use.

rene

Like No copyright infringement intended ?

Rob

10:25 AM

de minimis non curat lex

== English == === Etymology === From Latin dē minimīs nōn cūrat lēx. === Phrase === de minimis non curat lex (law) The law does not concern itself with trifles; expression of the rule that the law will not remedy an injury that is minimal. ==== Derived terms ==== de minimis === References === de minimis non curat lex at OneLook Dictionary Search de minimis on Wikipedia.Wikipedia

no harm, no foul

== English == === Etymology === From sports. === Pronunciation === === Phrase === no harm, no foul Although technically a breach of some code or law may have occurred, there was no actual damage meriting punishment, apology or retribution. He parked in my space, but I was away at the time: no harm, no foul. ==== Usage notes ==== Phrases with the same construction (i.e. no x, no y) are common and productive. See the related terms below. Compare the expression de minimis non curat lex (“the law will not cure (or concern itself with) trifles”), de minimis. ==== Synonyms ==== NHNF ...

If everyone stole copyrighted material there'd be no incentive for people to do work that had no renumeration; if the works truly have no value then why take them - is the infringer a binner?

Mithical

3:05 PM

→ 11 messages moved to Chimney

terdon

6:57 PM

I'm not? Since when you scurvy scalawag!?

Aibobot

I think cause you're on meta

terdon

Yeah, it worked in Charcoal HQ

Presumably my meta chat user doesn't have privileges. Sigh. One more annoyance of having multiple chat servers, add it to the list! :P

Actually, @Aibobot, could you spam flag that?

The post had been edited by a well meaning but misguided user so it doesn't currently look like spam which is probably why it has survived so long.

Aibobot

done

terdon

Cheers

Sigh. It's still there :/

Aibobot

People get annoyed if I double flag :D

terdon

7:04 PM

with good reason! :P

Aibobot

Its at -7 now though. I guess someone downvoted instead of flagged

terdon

more than one, most probably

certainly, even

Probably because the well-meaning but misguided user mentioned above edited the spam out leaving a place holder.

Nord The Lofty Wizard

10:47 PM

just curious but, any way i can format this post better? (to be more readable?):

0

Q: Automating vncserver with expect not working as expected

I thought of automating the part when i first run vncserver, since it ask for a password (and while i know there a flag to pass a passwd file, i prefer to generate a new one every time i need/want to): #!/bin/bash read -s PWD /usr/bin/expect -d <<EOF spawn /usr/bin/vncserver -geometry 1366x768 ...

bash vnc expect

feel like it would look weirder if i added headers to it

Smitop

is it a bug that there is a way for anyone to see the mod announcements?

Spevacus

11:39 PM

@Smitop I would think so, yes.

Transcript for