Category Archives: Code

How much should you customise?

Joel Spolsky has started a series on the making of FogBugz 4.0, initially talking about the requests of customers, and how (and when) this should translate into changes in your software.

The request for customisation is a question most of us involved in coding will face at some time or another. Even if your system has only a single “client”, do you really want to bend it and stretch it in such a way that it’s no longer future-proof?

In most cases, if you can avoid it, I suspect the answer is no. You want to keep it generic. Because as sure as night follows day, the pendulum of your customer’s direction will swing back the other way, and you’ll end up having to build yourself out of that hole they urged you to dig.

For example, in my work on the earlier version of what has become eVision’s Message Exchange, at times we have needed to generate and transmit messages quite different from the established standard types we were already spitting out. Natively we could handle XML and flat file (delimited or fixed-length) output files, but what if we wanted to add Zip compression on top of that? Or what about some other weird and wonderful thing?

In the case of Zip, we built that into the software using the very handy InfoZip DLL. But for others that appeared to be so specialised that we didn’t want to lumber the code with them forever more, we built on a “post-process” step, which allowed the output file to be subject to an external process before it was transmitted.

This kind of generic hook into the software has opened up numerous possibilities. So when the business dudes came along and asked for a file to be generated that did an additional lookup off a reference table, we were able to write that step as a separate process. It all works rather well.

So in some cases, the answer to customisation is interopability. Rather than bend and stretch your masterpiece into something it shouldn’t be, you put the hooks in so you or somebody else can do the bending and stretching over in a separate corner.

Your code can stay clean and future-proof, and everybody’s happy.

Inaccuracies reading Excel via ADO

Yesterday’s discovery with reading Excel via ADO and the ODBC Text Driver: numeric values may be wrong if it takes a guess that the column contains date fields.

The geeky detail (found in VB6SP5, Excel 2003, ADO 2.7, Jet Provider 4.0):

The text driver looks at the first few lines (configurable via the MaxScanRows setting or the Rows To Scan field in the DSN configuration box) to figure out what sort of data it is dealing with in each column. If there are numbers, it assumes the columns are numbers.

But if it sees dates, and sees no other data in the next few rows, then it assumes all the data in those columns are dates. If it gets further down to rows containing actual numbers, it still thinks it’s reading dates, and when it tries to convert the number to a date and back again, it causes a rounding error and ends up adding 1 to the final number.

In my case, the solution was to get the spreadsheets to contain zeroes (quite valid for the data being loaded) in the first few rows, instead of blanks. A little tricky, but it remains a good method for getting data out of Excel.

The bandwidth hogs at allresearch.com

It seems like some others my sites are being bombarded with hits from a mob called AllResearch. Apparently one of the things they do is hit RSS feeds and suck down every page referenced, for some kind of indexing. Judging from the amount of traffic they’re burning up, they suck big-time, in fact. I mean, indexers usually put in a lot of hits on web sites, but these guys are hitting more than 10 times as much as the next one down the list, MSN.

These are the top hitters over at toxiccustard.com:

  • 45541 sp1.allresearch.com
  • 3448 msnbot.msn.com
  • 3110 index.atomz.com
  • 1328 crawl25-public.alexa.com

Time for a little .htaccess magic:

order allow,deny
deny from 38.144.36.
allow from all

Ad blocking begins to have an economic effect

So I was checking out copper (as you do), and followed the wikipedia copper entry link to EnvironmentalChemistry.com’s copper data, and I discovered that ad blockers are beginning to change the economics of the web. The web site whinged that they had detected ad blocking, and if I wanted to get the content I’d have to turn it off (and provided directions – which I followed, but it just turned out to be a bunch of atomic numbers and covalent bonds and useless crap like that).

The economics of a lot of the web are not dissimilar to those of free-to-air television; there’s a covenant between the producers (broadcasters/webauthors) and the consumers – we will let this stuff out to anyone, and you will consume our advertising. Advertisers give the producers cash to cover the costs of publishing. There’s a profit in it, and everyone’s happy.

Except that consumers have decided they don’t like the deal anymore. People are taping TV shows, and skipping the ads. People are using ad blockers in their browsers. The economics of the model are breaking down. I personally am behaving this way because I find the advertising increasingly intrusive and irrelevant, and thus annoying. The ads suck, for products that suck, and they’re shoved down my throat. So I avoid them. This is how a character in Carl Sagan’s novel Contact became the richest man on earth – by selling TV ad blockers.

The three outcomes I can forecast from this are:

  1. increased relevance of advertising (unlikely, the reason advertising is necessary is because of an inherent suckiness of the products, otherwise they’d be compelling)
  2. decreased expenditure on content provision (on TV, cheaper nastier shows – if that’s possible; on the web, uneconomic sites being pulled or at least not updated)
  3. product placement, which is a bit like 1, ‘cept different because it’s more about appropriate products in appropriate places

I for one have no idea how this will play out, but I’m sure advertising will get more subtle. It’s done that over the last century, and will continue to in response to increasing consumer sophistication. Perhaps advertisers will find a way to back off, and only offer their products to customers who want them; they certainly want to act that way, because it’s a waste of money advertising women’s sanitary napkins to the gay male viewers of Friends — unless they’re planning to fix their car’s leaky roof with one.

BTW, how did they figure out I was blocking their ads?

Programming quote

This is possibly my favourite programming quote of all time:

“You know, when you have a program that does something really cool, and you wrote it from scratch, and it took a significant part of your life, you grow fond of it. When it’s finished, it feels like some kind of amorphous sculpture that you’ve created. It has an abstract shape in your head that’s completely independent of its actual purpose. Elegant, simple, beautiful.

“Then, only a year later, after making dozens of pragmatic alterations to suit the people who use it, not only has your Venus-de-Milo lost both arms, she also has a giraffe’s head sticking out of her chest and a cherubic penis that squirts colored water into a plastic bucket. The romance has become so painful that each day you struggle with an overwhelming urge to smash the fucking thing to pieces with a hammer.”

— Nick Foster (“Life as a programmer”)

(via Owen)

ASP.Net first impressions

I’m enjoying mucking about with Asp.Net. For someone who’s done a fair bit of “Classic” ASP like myself, it’s a great step forward.

But maybe I’m way off the mark here, how on earth could they implement something as useful as a DataGrid HyperLink column or a <asp:hyperlink> tag and not include an easy way of setting the title attribute in the link? Nor does the syntax checker seem to recognise the <span> tag. Did the W3C go and depreciate these while I wasn’t looking? Don’t believe so…

(The solution for the DataGrid is to use an <ItemTemplate> and code it yourself.

Ditto the Hyperlink tag. For the <span> tag, well, what care I if the syntax checker puts a squiggly line underneath it and it shouts a warning during compilation. Ah, seems to allow <span> and other such niceties, you have to tell it your targetting IE5 (or later), rather than IE3/NS3. Makes sense.)

75 digits of pi

(Does this count as a podcast?)

When I was a junior geek of 14 or so, some friends and I spent some time filling dead time in a maths class by learning digits of pi. I got to 75. Twenty years later, it’s still hanging about in my brain, wasting valuable brain cells.

Thank goodness it’s knowledge that is useful, rather than some pointless weird-arse geek party trick.

Click here to listen to 75 digits of pi. (171Kb, MP3, 21 seconds)

Office Object Library problems

One of the machines a program of mine runs on is still NT4 and Office 97. It seemed to keep working okay when I was on Win2K/Office 2000, but now I’m on WinXP/Office 2003 it’s crashing when calling the Excel object library to open an XLS.

MS have documented that this can happen after re-compiling with the Office 2000 libraries or later (though I’m sure it didn’t happen with Excel 2000).

“This behavior is by design.” Application errors by design. Yeah. SURE.

Excel application error

Apparently the solution is to use that normally-considered-evil late-binding; DIMming as an Object and using CreateObject.

But it didn’t seem to work for me. So I dug out an old copy of Office 97, installed it into a separate directory to Office 2003. Removed the reference to Excel 11, added Excel 8 instead (EXCEL8.OLB), recompiled and all is well.

The joys of .htaccess

For those who merely dabble in Apache, .htaccess seems a little like black magic. Yet it’s so useful… it can do default (index) documents, redirection, password protection, custom 404s, blocking image stealers… everything! This set of pages serves as a useful tutorial for doing it all.

Excel to VB bug?

From my limited testing (I had to find a workaround pronto) seems to be some kind of bug when using VB6, ADO 2.7, ODBC text driver to read from an Excel spreadsheet. Recordsets with date fields come in with the dates encoded as integers. This is normally no problem, as you can use IsDate to check if it’s valid, then CDate to convert it to a date.

But what I’ve found is that IsDate has stopped working… that is, it’s stopped returning True for field values that can be converted to dates, eg CDate works. Whether this is something to do with the new year, or a new version of something on my machine, I haven’t yet figured out, but I ended up writing a wrapper IsDate function that just tries the CDate regardless, and returns a False (eg not a date) if it errors.

An initial dig in the MSKB found nothing about it, but I’ll do some more exhaustive digging and try to get a definite answer later. The answer being, I suspect, “Switch to VB.Net, you luddite!”