Tavern on the Meta

Luke

12:13 AM

It's so quiet...

@TimStone What's "Happy Hour"?

Scrooginator

1:09 AM

Do want: japaneseknifecompany.com/KNIVES/HANDMADE/SAJI129LAYERCOLLECTION/…

Luke

2:15 AM

@TimStone Do you believe we are discriminating against unicorns also?

animuson

4:01 AM

Anyone around to snipe-reject some edits?

Two edits for a tag that has 0 questions: 1 2

animuson

6:00 AM

I've developed an odd habit of holding both shift keys when I want to capitalize something... :/

Moshe

7:49 AM

Anyone up for some PHP and XML parsing?

user200500

Sure

Pekka

9:06 AM

@Scrooginator oooooh. Me too.

Moshe

9:52 AM

Hi folks.

@Asad Still here?

I'm having fun.

user200500

10:12 AM

Yeah. Sort of in the process of writing a treatise on CSS rotate transformations.

user200500

0

A: cannot transition css3 rotate3d property from one specific orientation to another

The transforms occurring here are compound transforms (in that the transform matrices undergo matrix concatenation). When you rotate the element around its x-axis, for example, the other axes become rotated by 90 degrees, so that rotation about the y-axis no longer has the right to left effect as...

Moshe

10:48 AM

lol

@Lix - Are you around?

Lix

11:10 AM

hey @mos - I am... but can't really sit in char ATM...

Moshe

@Lix kk Got a git username that I can use to add you to a repo? I'm working on something and my PHP skills are rather lacking.

user200500

@Moshe What are you having trouble with?

Moshe

I'm trying to scrape a site. It kinda works, but I'm seeing two issues.

1. Incomplete parsing - I blame badly formed HTML.

2. Incomplete storage - I have no clue how to use an array or collection as a property of a PHP class.

user200500

11:25 AM

Are you using the native DOMParser to parse it?

Moshe

@Asad Yes.

user200500

Sorry, DOMDocument. And you're using loadHTML, not loadXML, right?

Moshe

@Asad yes

I'm pulling in an "HTML 4" page from a Uni website.

user200500

Your HTML must be pretty broken then, because that is supposed to be fault tolerant.

Moshe

@Asad You're telling me?

user200500

11:28 AM

Can I see the address?

Moshe

@Asad Sure, but I'll do one better - got a github name? I can add you to the repo which has the url and my code.

user200500

Nope. :D

Moshe

:-/

@Asad Its free to sign up.

user200500

I've been meaning to get into Git but at this point I don't even know how it works.

user200500

The whole revision control thing

Moshe

11:30 AM

ok, got an email addy?

got it

user200500

Is it throwing any exceptions?

Moshe

@Asad sent

@Asad nope

user200500

Alright, I'll have a look.

Moshe

It just stops parsing after a bit of elements.

@Asad thanks

user200500

Seems to be taking a while

Moshe

11:36 AM

@Asad Yes, it's slowish. I blame a combination of my amateur PHP and the school's server.

@Asad does it run?

user200500

Haven't even got it yet. I'm periodically checking both the main inbox and spam, I'll let you know when it gets here.

Moshe

@Asad Oooh, ok.

user200500

lol are you sure you got the address right?

Moshe

Let me know when you have it.

user200500

K got it.

Moshe

11:44 AM

@Asad Follow the link from here to see what I'm scraping.

@Asad power.php is the main file.

Although it may be obvious based on the contents of the other files.

user200500

Yeah I figured from the includes.

Moshe

So the curl request pulls the page, but the parser chokes somewhere and dies after a class called CORC 1100

and also, it doesn't store course sections.

@Asad any thoughts?

user200500

Alright I don't have curl compiled into my PHP, so I can't test this myself. You're sure the response is getting through, you clean it (assuming that isn't breaking the HTML), then it breaks when you loadHTML it.

user200500

Correct?

user200500

Also, take out the internal errors bit so you can actually see if any warnings get thrown.

Moshe

11:58 AM

@Asad I load it, clean it, it runs until a certain point. It breaks in the middle, but fails silently, so the printout prints some of it.

@Asad I did that.

No warnings.

@Pekka - hi

Pekka

Hi!

Moshe

@Pekka Working on some PHP code.

Pekka

Yeah I noticed

Moshe

Trying to scrape a uni course listing site.

So I can be awesome and cool and do some fun stuff.

But an IT dept thats 20 years behind is making my life less easy than it should be.

Also, I should have slept so I can study math.

I'm in trouble now.

Pekka

Tell them they need to provide an API

jk

Moshe

12:00 PM

Final on Tuesday.

user200500

Is it elementIsACourseSectionTable that breaks?

Pekka

That sounds difficult!

Moshe

@Pekka I hear they're doing something for some company. But I intend to show them that students aren't as dumb as they think.

Pekka

Heh. I see

Moshe

@Asad No idea what breaks, but I'm not seeing any course sections. commenting it out doesn't fix anything though.

@Pekka It's fun, having not done PHP in a long time. :-)

Also, student government was supposed to work with me, until they decided not to. Flip floppers gona flip flop.

user200500

12:02 PM

Here's the thing, if you dump $dom->saveHTML() and it is identical to what you fed the parser, we know the parsing isn't an issue.

Moshe

@Asad how so? var_dump?

/me ducks

warnings!

user200500

Yeah, preferably in pre tags to preserve any whitespace that might have come with it.

Moshe

@Asad Warning: DOMDocument::loadHTML() [domdocument.loadhtml]: htmlParseEntityRef: no name in Entity, line: 285

Loads of those.

user200500

TO THE GOOGLE

Moshe

ugh

user200500

12:05 PM

5

Q: Warning: DOMDocument::loadHTML(): htmlParseEntityRef: expecting ';' in Entity,

$html = file_get_contents("http://www.somesite.com/"); $dom = new DOMDocument(); $dom->loadHTML($html); echo $dom; throws Warning: DOMDocument::loadHTML(): htmlParseEntityRef: expecting ';' in Entity, Catchable fatal error: Object of class DOMDocument could not be converted to string in t...

php

Moshe

Found that too

user200500

That isn't it :(

Moshe

nope

I need sleep though

been up since 4 yesterday afternoon

user200500

Ok try running htmlentities on the HTML before you feed it to loadHTML

user200500

$dom->loadHTML(htmlentities($html));

user200500

12:09 PM

Sounds like they might have unescaped ampersands somewhere in the page

Moshe

@Asad Fatal error: Call to a member function getElementsByTagName() on a non-object in /home/content/44/4944444/html/lightbulb/power.php on line 109

nope

user200500

I don't get it. You've already instantiated the DOMDocument, it isn't like it evaporates even if you feed it invalid HTML. How is $dom a non object?

Moshe

@Asad no idea

user200500

It doesn't complain about calling loadHTMLon the same non-object though. You've already removed libxml_use_internal_errors(true); haven't you?

teresko

1:05 PM

hi everyone

i got hit yesterday a bit by serial downvoter

could it be that the bot still has not made it's rounds ?

user200500

Not sure. You could try flagging one of the posts that was hit and explain the situation.

teresko

1:25 PM

ok , flagged

Moshe

4:41 PM

@Lix - ?

Scrooginator

5:13 PM

Fix all the things.

Lix

5:33 PM

@mos - !

Moshe

6:25 PM

@Lix DOMDocument is truncating!

$dom = new DOMDocument;

libxml_use_internal_errors(true);
$dom->loadHTML($html);
libxml_use_internal_errors(false);

//
// Get all of the tables in the page
//

$tables = $dom->getElementsByTagName('table');

echo "There are " . $tables->length . " tables. \n\n\n\n\n";

Expected results: ~3K

Actual: 735

@TimStone You know anything about this?

Tim Stone

Is $html well-formed? And why does it contain 3K tables? >_>

Moshe

@TimStone It's nearly well formed. I do a lot of sanitizing.

There's some weird non-ascii characters in there, according to the W3 parser.

@TimStone And the tables come from a college database of course listings.

@TimStone - How would I see what's in the DOMDocument?

Tim Stone

Not entirely sure what you're asking. You can resave the DOMDocument to file or recursively traverse the child elements.

Moshe

@TimStone I want to verify that the DOMDocument loaded everything correctly, because there's significantly fewer tables in there than should be.

Tim Stone

6:42 PM

Yeah, then do one of those two things.

Moshe

ok

thanks

Tim Stone

Did you check if you were getting parse errors with libxml_get_errors() by the way?

Moshe

@TimStone I was getting some empty Entities.

Luke

I hate to break up a serious conversation, but does anybody here know what 'happy hour' is?

Lix

As in the happy hour here in the tavern or in general?

Luke

6:50 PM

In the tavern

Lix

It's just a recurring event that's called "happy hour".

Scrooginator

It's the special time on Friday where we all get a chat notification, realize it was just happy hour, talk about drinking for two minutes, then go about our usual business.

Moshe

@Lix

you're here

stay awhile

Scrooginator

stay FOREVER ah ah ahhhhh

Luke

Aha

Lix

6:52 PM

@mos I'm here

Moshe

@Lix Good, I'm trying to get this PHP to work.

Scrooginator

Now I'm feeling all nostalgic:

►

Lix

Haven't used DOMDocument that much... but i'll help where i can :)

Tim Stone

...is my answer to that question.

Moshe

@Lix It's reading 735 out of 2998 tables.

Lix

6:54 PM

What file are you reading that has that many entities?

animuson

That's a lot of beer...

Moshe

@Lix My school's course registration listing.

animuson

It's like walking into the room and being hit in the head with a bottle of beer...

Luke

Ouch

Tim Stone

@animuson And coffee!

Moshe

6:56 PM

So @ircmaxell mentioned Pecl.

Anyone know how to use that?

Google takes me to setup tutorials.

Lix

PECL is an extension library...

Moshe

Right, where's the docs?

Lix

for what extension? :P

Luke

@TimStone Do you believe SE discriminates against non-human life-forms?

Tim Stone

I'm secretly just a maroon and white square, so...no.

Lix

6:57 PM

@luk - much...

Anna Lear

@Luke They're nice to cats.

Moshe

@Lix the one that will clean this CUNYform HTML.

Luke

Has SE ever hired a non-human life-form?

animuson

Well, if you consider disembodied floating heads to be non-human...

Luke

I believe they are discriminating against unicorns.

Tim Stone

6:59 PM

@Luke @AnnaLear is apparently a feline Community Manager, so it would seem so.

animuson

They also employ gerbils...

Luke

After all, wasn't the Community user de-unicornified?

@animuson That's a little better...

animuson

And they have that lovely panda as their error spokesperson!

Anna Lear

@animuson And hamsters, especially in the sysadmin area.

Luke

So, they employ rodents, humans, and bears

What about lions and tigers?

Anna Lear

7:02 PM

Also, Grace Note is a cactuar.

Tim Stone

And an actual note:

Luke

I never knew nick plays cricket

animuson

@Luke I'm not sure Rebecca poking those two with pointy sticks would go over very well...

Luke

Moshe

So tidy works!

Lix

7:04 PM

@mos - was only a problem of malformed markup then?

Moshe

@Lix Likely.

animuson

The mail man came today... Does he know it's Sunday?

Anna Lear

@animuson maybe he really likes you?

animuson

He woke me up with the doorbell... so I'd say no.

Lix

probably only brought bills too...

Maybe some stickers...

Tim Stone

7:09 PM

Hmm, that's an interesting layout bug...

Moshe

yay!

Scrooginator

derp

Lix

(* removes the never ending gif animation *)

animuson

status-declined

Luke

7:26 PM

How did you remove it, @Lix? I still see it.

Lix

Chrome developers tools :P

Right click it, click inspect element, then slam down your favorite delete key...

Scrooginator

Or replace it with a more tolerable lolcat.

Luke

How dare you?

Lix

A gif like that I could stare at all day...

the one box doesn't do it justice...

cinemagraphs.com

Luke

animuson

7:34 PM

Ok, let's not get carried away... No one in here enjoys coming back to 2 screens full of images...

Luke

Lix

I ♥ chrome dev tools

Scrooginator

@animuson There's an app for that.

bah

Lix

awwww :/

animuson

What's with the extraneous exclamation point in front of the URL in the star list?

Is that new to identify it's an image or something?

Scrooginator

7:39 PM

It's to identify that the original url I tried posting which didn't end in .jpg was an image.

Now it's just redundant.

Moshe

8:15 PM

GoDaddy sucks

because they won't install Pecl for me.

Not because they have no ports open.

Not because they make everything difficult.

Not because of that inexcusable outage this semester.

But because they won't enable my script.

doofuses

ThiefMaster

That's why you avoid shared hosting.

Scrooginator

8:45 PM

Here use my referral link for some other shared hosting :P

Tim Stone

9:01 PM

in Chat feedback, Nov 8 at 4:04, by Tim Stone

If a leading ! is used for image-fication purposes, can it be hidden in the star list since it serves no purpose there?

user200500

Did you know facebook's started logging what you search? You can see it in your activity now.

user200500

@Moshe Did you figure out what the problem was?

Luke

acoustics.salford.ac.uk/acoustics_info/duck/media/…

The duck has quite a voice

Scrooginator

I just wanted to say...

Luke

Nothing?

Scrooginator

9:10 PM

I'm amazing.

Luke

Is my awesomeness making you speechless?

Scrooginator

Apparently that got rate-limited.

Luke

rioleo.org/mv-cahaba.php

Luke

10:25 PM

Do the hamsters get paid by the hour or are they salaried?

Pekka

10:44 PM

It's more complicated than that. They get a small base pay, but most is paid in performance-dependent bonuses

Luke

11:02 PM

@Pekka Do you believe SE is abusing these rodents?

Moshe

I did it.

@Asad Oh, yea. malformed stuff.

But GoDaddy wouldn't let me run Tidy

so I signed up with someone else.

Luke

11:37 PM

@Pekka What is the average weekly pay check per hamster?

Transcript for