Category Archives: Data

Allow more JavaScript, maintain privacy

I’ve long regarded JavaScript in the browser to be one of the biggest security holes in web-browsing, and at the same time the Internet works less and less well without it. In 2008 Joel Spolsky made the observation that for some people the Internet is just broken:

Spolsky:   Does anybody really turn off JavaScript nowadays, and like successfully surf the Internets?

Atwood:   Yeah, I was going through my blog…

Spolsky:   It seems like half of all sites would be broken.

Which is not wrong.  Things have changed in the last five years, and now the Internet is even more broken if you’re not willing to do whatever random things the site you’re looking at tells you to, and whatever other random sites that site links off to tell you to, plus whatever their JavaScript in turn tells you to. This bugs me because it marginalizes the vulnerable (the visually impaired, specifically), and is also a gaping security hole.  And the performance drain!

Normally I rock with JavaScript disabling tools and part of my tin-foil-hat approach to the Internet, but I’m now seeing that the Internet is increasingly dependent on fat clients. I’ve seen blogging sites that come up empty, because they can’t lay out their content without client-side scripting and refuse to fall back gracefully.

So, I need finer granularity of control.  Part one is RequestPolicy for FireFox, similar to which (but not as fine-grained) is Cross-Domain Request Filter for Chrome.

The extensive tracking performed by Google, Facebook, Twitter et al gives me the willys. These particular organisations can be blocked by ShareMeNot, but the galling thing is that the ShareMeNot download page demands JavaScript to display a screenshot and a clickable graphical button – which could easily been implemented as an image with a href. What the hell is wrong with kids these days?

Anyway, here’s the base configuration for my browsers these days:

FireFox Chrome Reason
HTTPSEverywhere HTTPSEverywhere Avoid inadvertent privacy leakage
Self Destructing Cookies “Third party cookies and site data” is blocked via the browser’s Settings, manual approval of individual third party cookies. Avoid tracking; StackOverflow (for example) completely breaks without cookies
RequestPolicy Cross-Domain Request Filter for Chrome Browser security and performance, avoid tracking
NoScript NotScripts Browser security and performance, avoid tracking
AdBlock Edge Adblock Plus Ad blocking
DoNotTrackMe DoNotTrackMe Avoid tracking – use social media when you want, not all the time
Firegloves (no longer available), could replace with Blender or Blend In I’ve have had layout issues when using Firegloves and couldn’t turn it off site-by-site

Summer 2013/2014 starts

The current 7-day forecast for Melbourne:

Friday   30 August           Max 20 Shower or two.
Saturday 31 August    Min 12 Max 23 Sunny.
Sunday    1 September Min 15 Max 25 Partly cloudy.
Monday    2 September Min 12 Max 23 Partly cloudy.
Tuesday   3 September Min 11 Max 25 Partly cloudy.
Wednesday 4 September Min 16 Max 26 Shower or two developing.
Thursday  5 September Min 16 Max 20 Shower or two.

I declare summer whenever there’s going to be 7 consecutive days in a row above 19 degrees.  Previously, the earliest Summer has started was mid-September, but typically it’s been moving forward from October or November.

Remember we’ve got an election coming up in a week’s time, and that’s your opportunity to repeal the carbon tax.  Which we need to do, to keep lovely balmy weather happening in winter-time and to keep the cost-of-living down.  Remember: carbon-dioxide is food for plants, and as such good for the environment, which is made out of plants. That’s just science.

Flooding with water

So, looking at properties, and a number are down on the floodplain near the local moving body of water, a river/creek.  I wonder to myself if the area is at any risk from floodwater; should I even bother looking at the area?

The council, being the government body most connected to the area, ought to know.  It doesn't; it can't tell me except to tell me if a specific property has a flood-overlay, which says that modelling has determined that it is at risk of a 1 in 100 year flood.

What is the 1 in 100 year flood event?

The 1 in 100 year flood event is the storm that happens on
average once every one hundred years (or a 1% chance of
occurring in any given year).

Now, that means in any given year there's a 99% chance you're not going to get flooded.  In 100 years, that means a 0.99100 or a 36.6% chance of not getting flooded. A 2/3 chance of having water washing through your home at some point there.  Basically, that's a guarantee that in the next century your home will be damper than normal – because the 1 in 100 year events are calculated off historic data, not forward climate models.  And the forward models say that things are only going to get more extreme; have you noticed how 1 in 100 year events seem to happen to the same place every decade or so?

In fact, pretty much anyone you talk to – water utilities for example – will only talk about 1 in 100 events. Vital government infrastructure (stuff that has to keep operating the event of a flood disaster, like hospitals and my home) has to be above the 1 in 500 line. From what I'm told, they calculate this on a site-by-site basis rather than having a map (they're not building a bunch of new hospitals, so it's easier that way).  Sites aren't rated as being 1 in 110 year, you're either in the 100 year box or not rated at all.

The gist of what I was able to read into the subtext of the hints being passed in my conversation with a town planner specializing in flooding was: Floodplains get flooded, even in cities, even if there's a wetlands further upriver that could absorb a sudden influx of water, even if the sides of the creek are quite steep and the channel is surprisingly broad, and even if there are barricades; If you don't like that, don't live there.

So I won't.  It makes searching for a home so much easier, even if the homes out of the floodplain are more expensive and built on those annoyingly sloped hill things.

Actually, this reminds me of the 1972 Elizabeth St Floods my Mum told me about getting caught in. I would never have guessed a major street in our CBD could turn into a river – and then it happened again in 2010.

zp8497586rq

Summer 2011/2012 starts

I declare summer whenever there’s going to be 7 consecutive days in a row above 19 degrees.  And as such:

Friday            Max 22  Shower or two.    
17/09/11  Min 10  Max 25  Shower or two developing.    
18/09/11  Min 12  Max 21  Sunny.    
19/09/11  Min 11  Max 27  Showers developing. Windy.
20/09/11  Min 12  Max 20  Shower or two.     
21/09/11  Min 13  Max 22  Morning shower or two.    
22/09/11  Min 12  Max 24  Mostly sunny.

Summer.

In September.

It’s a good thing that global warming is a beat up by the greens, a front for communist interests trying to take control of our lives and introduce excessive and unneeded regulation – or else I might be worried.

Census night is coming

The census delivery chick turned up and offered us the option of paper or electronic form.

Two programmers looked at each other, thought about how they value their time and the response was a no-brainer:

“We’re programmers,” I explained, “we’ll take the paper form.”

“There’s a phone number you can call if you have any trouble filling out the electronic form” reassures the collector.

Cathy thinks: “Sure, that line won’t have any trouble when twenty million Australians simultaneously log into the web site to fill in the forms via a broken SSL link, using IE specific controls (that only work under some versions Windows assuming they’re correctly patched and have the right libraries loaded), demanding full round-trips to the underspec’d Windows servers to populate unnecessarily complex custom controls, some of which will no doubt demand Flash or COM. Come to think of it, it probably won’t even be web based, and we’ve only got two Windows boxes, one of which is tucked under a table (Yay! Census night on the floor swearing at the ABS’s programmers!) and the other has a screen resolution that went out with buggy whips (I’ve had programs barf and refuse to run because the resolution was unacceptable).”

We chose paper. For another view of the world, I’m looking forward hearing to how census night worked for Daniel…

Importing into SQL Server

Alas SQL Server Management Studio isn’t as friendly as it could be for pasting in data. You’d think Microsoft would have this humming, but when I tried to paste from Excel, it attempted to paste the entire first row from my spreadsheet into the first column (in one row) of the database.

Using MS Access to open up the database probably would have worked, but I didn’t have it on that machine.

Trying to import using the SQL Server Import And Export Data wizard from a CSV text file worked for a small amount of data, but the 80,000 rows I was trying to import from the world ports code list didn’t. Time and time again it would report an error (unspecified) and give me the option of Abort, Retry, Ignore. No matter option I chose, it crashed.

While the 64-bit version of the wizard on my 64-bit Win7 machine didn’t allow you to import from Excel/Access, the 32-bit version did (presumably because MS Office, at least the version I have installed, is 32-bit).

The next problem was that it only supported Excel 2003 format, which can’t handle more than 64K rows. I ended up having to split the data into two and import the two spreadsheets separately. Then it worked.

Shame the wizard is so flaky, and of course it’s a big shame that Management Studio doesn’t do copy/paste like one would expect. (Maybe that too was a 32-bit/64-bit issue.)

AU timezones

I’m not happy when I see someone technical quoting a time in summer (eg during daylight savings) which claims to be “AEST”.

It’s almost certainly actually AEDT.

A summary of the abbreviations, which looks reasonably official, is here: Australian timezones.

In summary, AEST, ACST and AWST apply in winter. AEDT, ACST and AWST apply in summer.

The other issue I had with a recent email was it said 12:00pm AEST. I think in this context, they meant midnight AEDT, but it’s confusing. Better to either say Midnight, or use 24-hour time: 00:00.

Facebook: Download your information

Facebook downloadI had a quick look at Facebook’s Download Your Information feature — evidently added a few months ago due to criticism about the accessibility of people’s data once it’s dumped into the Facebook bottomless pit.

You can find it via the My Account screen, by clicking Download Your Information.

It asks for some time to compile all the information — in my case this took about half an hour — then emails you to say it’s ready to download, and provides a link and re-checks your password.

It comes as a single zip file, with HTML and pictures inside it.

Opening the index.html file, you’ll find a version of your Profile page, with links to all the other information in the archive, including Wall, Photos, Friends, Events, Messages.

The Wall in my case was 1.5 Mb of HTML, going back to 2007, and I suspect is every Wall post (and replies from friends) I’ve ever made. Friends is just an unlinked list of all your friends (name only). Messages has all your message threads, and replies.

You can browse the photos via the directory of the same name; subdirectories reflect the folders. It looks like all the photo files are at the size that Facebook shrunk them down to when they were uploaded.

To actually get this information into another service, you’d need to do some trickery with munging the HTML. The code they’ve used seems relatively clean and easy to parse.

So all in all, quite a handy feature, and goes a long way towards dispelling fears that information pumped into Facebook was lost forever behind a zillion clicks of to show “Older Posts”.

(It doesn’t appear that Twitter has a comparable feature.)

Keeping old content

Unlike many organisations, the BBC has a very enlightened policy on leaving old content up on their web site.

Among other things, it says:

Our view is that these pages often contain a lot of information about the programme or event which may be of interest in the future. We don’t want to delete pages which users may have bookmarked or linked to in other ways.

In general our policy is only to remove pages where the information provided has become so outdated that it may lead to actual harm or damage.

If only more web sites took this view.

Powerpoint file sizes

Was dealing with a big Powerpoint presentation (PPT) file.

In the older PPT format, 6063 Kb.

When zipped, 4826 Kb. Not a bad saving given the number of pictures in it.

Here’s the interesting thing: in PPTX format: 3293 Kb.

Remembering that PPTX and other Office Open XML formats (DOCX, XLSX etc) do their compression on the file as a whole, not the individual componenets, so this is an interesting result.

Perhaps the old binary format is inherently less efficient/compressible than the new XML format.

Mind you, another big PPT I tried it with didn’t compress down as much; the PPTX was about the same size as the ZIPped PPT, so it obviously depends on the exact content

Disney: evil, but defeatably evil

Disney DVD’s slogan is Moves, Magic and More.  They got the more part right for sure.

There I am trying to back up my copy of Wall_e_lic2_d1 so that once the kids have scratched the living bejesus out of the playing version, a new one can be generated from the master. And also as to avoid the annoying ads, language selection and other remote-control-based activity at the start – just shove it in the DVD player and walk away. Thankfully Australian copyright law lets me do this.

The studio have been dicking around with the disk’s table of contents, giving it over seventy files it claims are five gig in size – which, giving the DVD specification, is not possible. What you need to do in circumstances like this is play it in some player that will tell you what the magic track that actually contains the movie, not some hacked version of it. Then back that one up.

In the case of this particular disk it’s track 53, 1:33:26 long weighting in at 5425.95MB in size.

Thing is, DVDShrink barfs on it. Like it does Cars, but for different reasons. Thankfully I’ve recently discovered that Linux has an equivalent to DVDShrink, but this one is still being maintained. K9copy is it’s name; Cars was processed with no problems, and it was only the tomfoolery on Wall-E that caused a pause in activity.

So there’s one less application that I need a copy of Windows to run.

Real Estate Websites Suck: Part 4

I’ve decided that I’m only going to look for properties with 4 (or more) bedrooms. I enter this as a search criteria, and the website says quite clearly “Results for properties for rent with 4+ bedrooms in {suburb}”.

So why do I get presented with 3 bedroom properties?

Facepalm. Five years, and these web sites still suck balls. Not only do searches not work, it appears that the site pegs my CPU at 100% when the rendered page is just sitting there. Some of their lovely JavaScript goodness I suppose.

If you ask nicely I might dig up and dust off my rant from five years ago…