Category Archives: Code

COM+ calling multiple .Net versions risks blowing the memory limit

From the Things I’ve Found That Might Affect Other People But There’s Not Much Detail In Google About It department

Not that I have much to add to the sum total of published knowledge on this, but our situation is we have a COM object (legacy code) calling .Net 2 objects (not-quite-as-old legacy code).

We’re upgrading it all to .Net 4.5, but the .Net stuff is an easier change, so we’ve been doing that first, naturally in a carefully planned, staged manner.

It turns out that COM+ objects have a 1 Gb memory limit.

And it appears that when you have them calling a mix of .Net 2 and .Net 4 objects, the overhead of being able to call two different .Net Framework versions becomes an issue.

At least that’s our theory based on what we’re seeing — looking at ProcessExplorer showed about a dozen assemblies loaded for each version of .Net.

One blog post I read suggested a 600-800 Mb overhead for each .Net version. I can’t confirm that, but certainly we’ve had more Out Of Memory exceptions than we expected.

We’ve resolved it by going back to our .Net 2 code (so only one version is held within the COM+ process at once), and we’ll do the big switch all at once. Hopefully that’ll resolve it.

The COM object will also be switched too of course, and once that’s done, running everything in .Net 4, plus a move from 32-bit to 64-bit machines, apparently will lift that 1 Gb limit to 8 Tb!

Cool!

SQL Server paranoia mode

This post is for Tony!

Worried about accidentally overwriting critical data with a typo in your database commands?

SQL Server Management Studio has an option for that: SET IMPLICIT TRANSACTIONS, which you’ll find (in 2008 / 2008 R2 at least) under Tools / Options / Query Execution / SQL Server / ANSI.

SQL Server implicit transactions option

Anything you do (from the next query window you open) will automatically be in a transaction, so you can ROLLBACK if you realise you’ve done the wrong thing.

Be warned, you’ll need to get into the habit of manually COMMITting everything. Don’t be tempted to just add the COMMIT at the bottom of your query… that would defeat the purpose.

It’ll prompt you do to so if you close a query window without having done a ROLLBACK or COMMIT. It can get a little irritating, but knowing you can’t accidentally trash all your data may give you piece of mind in return.

FOP and ZXing compatibility

From the I’ve been trying to get this to work — this tip may save you some time department:

Apache FOP is for doing graphics with XML and XSLT.

FOP version 0.95 came out in August 2008. 1.0 in July 2010. 1.1 in October 2012.

ZXing is for doing QR codes.

ZXing 0.1.2 was released in March 2012, and seems to no longer be in active development.

The important bit: As far as I can see, ZXing works fine with FOP 0.95 (which was current when it was released), but not with later versions.

I kept getting these errors:

SEVERE: Image not available. URI: (instream-object). Reason: org.apache.xmlgraphics.image.loader.ImageException: The file format is not supported. No ImagePreloader found for null (No context info available)
org.apache.xmlgraphics.image.loader.ImageException: The file format is not supported. No ImagePreloader found for null

With any amount of fiddling, I couldn’t get it to work on Windows and Java 1.7. Nobody online appears to have noted this issue. You might have more luck…

Or maybe there is no solution to it. If so, I hope this saves you some time.

(Barcode4J, which we also use, seems to work with all currently available versions of FOP. It also has QR functionality, provided via ZXing, but this is only available in the unreleased code, which presumably has alpha status, and you have to build yourself. Barcode4J also hasn’t been updated in some years, last released in December 2010.)

Unhelpful web help

Just… just… wrong. So wrong.

FlickrHelp
Firstly, note the error message “Enter a valid email addresss”. Where, pray tell, ought I do this?  Why do I need to upload any attachment again?  Why do I have to prove I’m a human time-after-time, when all I’m doing is wrestling with your completely broken attempt at a web form?

Have they noticed that no-one is submitting help requests via this form, what with its refusal to accept said requests?

Dear Flickr: stop sucking balls.

Political donations are not the problem

Corrupt politicians have recently been in the Australian news.

It has been observed that money, in the form of political donations, is a corrupting influence. This causes hand-wringing, as banning donations is considered to hinder the freedom of political expression.

As a response to this demand for cash to finance political expression, suggestions are made that private funding of politics be replaced by public funding – basically an increase on the funding which parties already receive (something of the order of $2.48 per primary vote in lower house seats in the last federal election, for example). This grates those with a strong dislike of politicians and the political process. In addition, the current funding model of retrospective funding (based on votes received) disenfranchises new political views – it locks in the existing players by funding them, allowing them to campaign for votes that will fund them; those outside the system will not be able to break in.

To allow new entrants into the political system to be funded on an equitable basis, some kind of on-going polling could be done and a funding stream allocated on proportionate support in non-electoral polls.

However, switching to purely taxpayer-funded funding isn’t necessary, even if in effect the tax-deductibility of political donations makes them taxpayer subsidised.

Political donations are not the problem, the problem is that donors can be identified by the political party and and expectation of quid pro quo is raised. Beyond that, large donations from a single donor are also a problem – even if political party donations were anonymised and repudiable the donation’s existence could be inferred by the velocity of money flowing out of any anonymising system.

Let’s say you’re trying to run a corrupt political party under an annoymised donation system. Someone comes to you and says “I will give your corrupt party $10m, and I expect you to make this corrupt thing happen.” You’d then donate the $10m, and your donation would be pooled along with the hundreds of other donations made to the party. The Donor Anonymising Service (DAS) would then hand over a certain amount of money to the party, but it would not be $10m. It would be the stipend that the party had requested from the DAS, along with advice that the current amount held in reserve is enough to last at least X days, where X was the same number (give or take a couple of days) as it was yesterday. You don’t know if the $10m donation was actually made, all your party knows is that it’s got enough money to last X+2 days. You could up the rate of the stipend, but the DAS would scale back the reported window so that no extra information is revealed by the reported minimum duration the reserves will last. You’d limit the rate and number of times the stipend could be changed to discourage probing. Naturally, it would be illegal to make a political party aware of a donation or its amount.

Of course, then you have all the fun and games associated with loaning money to political parties, and with corrupt administration of a Donor Anonymising Service, but you get the gist of where we could go with this idea.

BASIC turns 50

The BASIC programming language turned 50 on Thursday.

This Time article is a great read — and notes the importance of the language on getting school students programming.

What’s the equivalent today? The Raspberry Pi is helping make hardware affordable. Some might dabble in Visual Basic or C# via Visual Studio Express, or the many of the other freely available languages such as Python, PHP, Javascript… or here’s another way of looking at it, from Jeff Atwood:

Programmatically create Django security groups

Django authentication has security roles and CRUD permissions baked in from the get-go, but there’s a glaring omission: those roles, or Groups, are expected to be loaded by some competent administrator post-installation.  Groups are an excellent method of assigning access control to broad roles, but they don’t seem to be a first-class concept in Django.

It seems that you can kind-of save these values in by doing an export and creating a fixture, which will automatically re-load at install time, but that’s not terribly explicit – not compared to code. And I’m not even sure if it will work.  So here’s my solution to programmatically creating Django Groups.

management.py, which is created in the same directory as your models.py and is automatically run during python manage.py syncdb:

from django.db.models import signals
from django.contrib.auth.models import Group, Permission
import models 

myappname_group_permissions = {
  "Cinema Manager": [
    "add_session",
    "delete_session",
    "change_ticket",
    "delete_ticket",         # for sales reversals
    "add_creditcard_charge", # for sales reversals
    ],
  "Ticket Seller": [
    "add_ticket",
    "add_creditcard_charge",
    ],
  "Cleaner": [ # cleaners need to record their work
    "add_cleaning",
    "change_cleaning",
    "delete_cleaning",
    ],
}

def create_user_groups(app, created_models, verbosity, **kwargs):
  if verbosity>0:
    print "Initialising data post_syncdb"
  for group in volunteer_group_permissions:
    role, created = Group.objects.get_or_create(name=group)
    if verbosity>1 and created:
      print 'Creating group', group
    for perm in myappname_group_permissions[group]: 
      role.permissions.add(Permission.objects.get(codename=perm))
      if verbosity>1:
        print 'Permitting', group, 'to', perm
    role.save()

signals.post_syncdb.connect(
  create_user_groups, 
  sender=models, # only run once the models are created
  dispatch_uid='myappname.models.create_user_groups' # This only needs to universally unique; you could also mash the keyboard
  )

And that’s it. Naturally, if the appropriate action_model permissions don’t exist there’s going to be trouble.  The code says: After syncdb is run on the models, call create_user_groups.

Where did I take that photo?

I couldn’t find anyone extracting out the geolocation geotagging EXIF data from their photographs so they could pull it up on something like Google Maps.  There are stand-alone programs with embedded maps, but the bits and bobs lying around on the average system ought to be enough to just generate a URL to a mapping website.  The following bash script echoes the  URL that geolocates your JPEG.  Because my camera doesn’t emit it, I couldn’t be bothered dealing with the seconds part of a location, but I did detect that you don’t have a camera the same as mine.  Drop a line if you’ve used this and fixed it.

#!/bin/bash
# emit a hyperlink to google maps for the location of a photograph
declare Seconds=""
Seconds=`exif -m --ifd=GPS --tag=0x02 $1 | grep -oP "[\d|\d\.]+$"`
if (( $Seconds=='0' ))
then
  Seconds=`exif -m --ifd=GPS --tag=0x04 $1 | grep -oP "[\d|\d\.]+$"`
fi
if (( $Seconds!='0' ))
then
  echo
  echo "Script does not support seconds being specified"
  exit
fi
echo -n "https://maps.google.com.au/?q="
declare NorthSouth=`exif -m --ifd=GPS --tag=0x01 $1`
if [ "$NorthSouth" == "S" ] 
then
  echo -n "-"
fi
echo -n `exif -m --ifd=GPS --tag=0x02 $1 | grep -oP "^[\d|\d\.]+"`
echo -n "%20"
echo -n `exif -m --ifd=GPS --tag=0x02 $1 | grep -oP "(?<= )[\d|\d\.]+,"`
declare EastWest=`exif -m --ifd=GPS --tag=0x03 $1`
if [ "$EastWest" == "W" ]
then
  echo -n "-"
fi
echo -n `exif -m --ifd=GPS --tag=0x04 $1 | grep -oP "^[\d|\d\.]+"`
echo -n "%20"
echo -n `exif -m --ifd=GPS --tag=0x04 $1 | grep -oP "(?<= )[\d|\d\.]+(?=,)"`
echo

Install mwparserfromhell on Linux

Here’s how to install mwparserfromhell on Linux:

sudo apt-get install python-dev
sudo apt-get install python-pip
git clone https://github.com/earwig/mwparserfromhell.git
cd mwparserfromhell
python setup.py install

After which, wikitools by MrZ-man is nice for power-users:
svn co https://github.com/alexz-enwp/wikitools
cd wikitools/trunk
sudo python setup.py install

Installing Pygal into Cygwin

Pygal is a python library for emitting SVG charts. It might do PNGs too; the documentation is… sparse. Okay, there’s no documentation, but they show you several ways to make bar charts, and figure you can follow on from there.  Anyways, the installation instructions don’t work, not under cygwin.

Here’s what you should do:

  1. ensure cygwin has the libs libxml2-devel and libxslt-devel installed
  2. issue the command
    cygwin$ pip install pygal

and you’re done. Getting pip into cygwin is a whole world of hurt, but you will need to go looking for a http (not https) source to download setuptools, then download and run ez_setup.py, followed by using pip to upgrade setuptools. Have fun with that; I know I did.

Allow more JavaScript, maintain privacy

I’ve long regarded JavaScript in the browser to be one of the biggest security holes in web-browsing, and at the same time the Internet works less and less well without it. In 2008 Joel Spolsky made the observation that for some people the Internet is just broken:

Spolsky:   Does anybody really turn off JavaScript nowadays, and like successfully surf the Internets?

Atwood:   Yeah, I was going through my blog…

Spolsky:   It seems like half of all sites would be broken.

Which is not wrong.  Things have changed in the last five years, and now the Internet is even more broken if you’re not willing to do whatever random things the site you’re looking at tells you to, and whatever other random sites that site links off to tell you to, plus whatever their JavaScript in turn tells you to. This bugs me because it marginalizes the vulnerable (the visually impaired, specifically), and is also a gaping security hole.  And the performance drain!

Normally I rock with JavaScript disabling tools and part of my tin-foil-hat approach to the Internet, but I’m now seeing that the Internet is increasingly dependent on fat clients. I’ve seen blogging sites that come up empty, because they can’t lay out their content without client-side scripting and refuse to fall back gracefully.

So, I need finer granularity of control.  Part one is RequestPolicy for FireFox, similar to which (but not as fine-grained) is Cross-Domain Request Filter for Chrome.

The extensive tracking performed by Google, Facebook, Twitter et al gives me the willys. These particular organisations can be blocked by ShareMeNot, but the galling thing is that the ShareMeNot download page demands JavaScript to display a screenshot and a clickable graphical button – which could easily been implemented as an image with a href. What the hell is wrong with kids these days?

Anyway, here’s the base configuration for my browsers these days:

FireFox Chrome Reason
HTTPSEverywhere HTTPSEverywhere Avoid inadvertent privacy leakage
Self Destructing Cookies “Third party cookies and site data” is blocked via the browser’s Settings, manual approval of individual third party cookies. Avoid tracking; StackOverflow (for example) completely breaks without cookies
RequestPolicy Cross-Domain Request Filter for Chrome Browser security and performance, avoid tracking
NoScript NotScripts Browser security and performance, avoid tracking
AdBlock Edge Adblock Plus Ad blocking
DoNotTrackMe DoNotTrackMe Avoid tracking – use social media when you want, not all the time
Firegloves (no longer available), could replace with Blender or Blend In I’ve have had layout issues when using Firegloves and couldn’t turn it off site-by-site