Category Archives: Python

Programatic submission of Australia Post’s CN23 customs form

A number of major international destinations of packages now will only accept packages with electronic CN23 customs declaration. Normally, you’d do this by rocking up to the Post Office with your pre-addressed parcel, filling in a CN23 paper form, and have that transcribed into Australia Post’s computer system by the postal worker behind the counter. You can elect to receive SMS notifications of change of status (landed, delivered, etc) for 50c.

Australia Post also allows you to fill in the appropriate details on their website; if you do this, then you get a QR code sent to you via SMS (free) and email (free) which the postal worker scans in and all the details (your name and address, destination name and address, contents, etc) are attached to your package’s details without any error-prone re-keying. The downside of going down this path is the dismal website Aussie Post provides, a JavaScript heavy, painfully slow dog of a site that doesn’t cache your own address.

Once the QR code is scanned, and the postal worker checks everything with you, they’ll print out the CN23, get you to sign it , and then it gets attached to your parcel. Because the To and From addresses are on the CN23 form (and those details are in electronic form, associated with the barcode for the package), it’s perfectly acceptable to present an unaddressed package to the post office (make sure you can tell which package is which, if you go down this route).

One thing you need to be aware of: Australia Post hasn’t heard of Unicode. You absolutely can’t use any characters not in the ASCII character set, and even then a very limited range of them. Certain fields allow some characters, which in turn aren’t allowed in other fields.

One of the fields you can supply is the HS tariff code, which is an international standard group of codes to describe “stuff” – the Harmonised System Tariff code. The sourcecode below uses the code for “Toy, plastic construction” – you should use the code for what you’re actually sending. You can specify multiple HS codes. Dollar values are in decimal dollars, weights are in decimal kilograms.

After calling the Australia Post website with your customs declaration, it returns to you a base-64 encoded PNG of the QR code to present at the counter, and a base-64 encoded PDF of the CN23 form – there’s no point printing this out, because it’s not paid for yet; let the Post Office print it out with the postage on it. You’ll also get the PNG via email and SMS (free).

Here’s some Python to make this submission:


    AP_session = requests.Session()
    jsonFormData =  {"customDeclaration":{
      "label":{"source":"AEM","postagePaidIndicator":False,"eadIndicator":False},
      "parcelCharacteristics":{
        "productClassification":11,
        "dangerousGoodsIndicator":False,
        "returnInstructions":"Return By Most Economical Route",
        "confirmationMobileNumber":"0411111111",
        "content":[{
          "content":"HS traffic code name for your stuff",
          "contentQuantity":1,
          "contentUnitValue":subtotal,
          "totalContentValue":subtotal,
          "contentWeight":int(order["total_weight"])/1000,
          "hsTariff":"95030039",
          "contentCountryOfOrigin":"DK"
          }],
        "totalConsignmentValue":subtotal},
      "senderAddress":{"firstName":"Josh","lastName":"FromGeekrant.org",
        "addressLine":["11 Example St"],"suburb":"YourSuburbName","state":"VIC",
        "postcode":"3000","email":"addr@example.com",
        "phone":"0411111111","smsConfirmation":False,"countryCode":"AU"},
      "receiverAddress":{"firstName":CustomsString(order["label_address_name_first"]),
        "lastName":CustomsString(order["label_address_name_last"]),
        "countryCode":order["label_address_two_char_country_code"],
        "addressLine":CustomsAddress(order),
        "suburb":CustomsString(order["label_address_city"]),
        "state":CustomsString(order["label_address_state"]),
        "postcode":CustomsString(order["label_address_postal_code"]),
        "email":order["buyer_email"]}
    }}

    stopact = {"jsonFormData":json.dumps(jsonFormData) }
    result = AP_session.post(url='https://auspost.com.au/bin/form/stopact', 
      data=stopact, timeout=2)
    response = json.loads(result.text)
    result.raise_for_status()
    filename = "{}-customsQRcode.png".format(orderid)
    with open(filename, "wb") as fh:
      fh.write(base64.b64decode(response['qrCode']))
    filename = "{}-CN23.pdf".format(orderid)
    with open(filename, "wb") as fh:
      fh.write(base64.b64decode(response['label']))

Programmatically create Django security groups

Django authentication has security roles and CRUD permissions baked in from the get-go, but there’s a glaring omission: those roles, or Groups, are expected to be loaded by some competent administrator post-installation.  Groups are an excellent method of assigning access control to broad roles, but they don’t seem to be a first-class concept in Django.

It seems that you can kind-of save these values in by doing an export and creating a fixture, which will automatically re-load at install time, but that’s not terribly explicit – not compared to code. And I’m not even sure if it will work.  So here’s my solution to programmatically creating Django Groups.

management.py, which is created in the same directory as your models.py and is automatically run during python manage.py syncdb:

from django.db.models import signals
from django.contrib.auth.models import Group, Permission
import models 

myappname_group_permissions = {
  "Cinema Manager": [
    "add_session",
    "delete_session",
    "change_ticket",
    "delete_ticket",         # for sales reversals
    "add_creditcard_charge", # for sales reversals
    ],
  "Ticket Seller": [
    "add_ticket",
    "add_creditcard_charge",
    ],
  "Cleaner": [ # cleaners need to record their work
    "add_cleaning",
    "change_cleaning",
    "delete_cleaning",
    ],
}

def create_user_groups(app, created_models, verbosity, **kwargs):
  if verbosity>0:
    print "Initialising data post_syncdb"
  for group in volunteer_group_permissions:
    role, created = Group.objects.get_or_create(name=group)
    if verbosity>1 and created:
      print 'Creating group', group
    for perm in myappname_group_permissions[group]: 
      role.permissions.add(Permission.objects.get(codename=perm))
      if verbosity>1:
        print 'Permitting', group, 'to', perm
    role.save()

signals.post_syncdb.connect(
  create_user_groups, 
  sender=models, # only run once the models are created
  dispatch_uid='myappname.models.create_user_groups' # This only needs to universally unique; you could also mash the keyboard
  )

And that’s it. Naturally, if the appropriate action_model permissions don’t exist there’s going to be trouble.  The code says: After syncdb is run on the models, call create_user_groups.

Install mwparserfromhell on Linux

Here’s how to install mwparserfromhell on Linux:

sudo apt-get install python-dev
sudo apt-get install python-pip
git clone https://github.com/earwig/mwparserfromhell.git
cd mwparserfromhell
python setup.py install

After which, wikitools by MrZ-man is nice for power-users:
svn co https://github.com/alexz-enwp/wikitools
cd wikitools/trunk
sudo python setup.py install

Installing Pygal into Cygwin

Pygal is a python library for emitting SVG charts. It might do PNGs too; the documentation is… sparse. Okay, there’s no documentation, but they show you several ways to make bar charts, and figure you can follow on from there.  Anyways, the installation instructions don’t work, not under cygwin.

Here’s what you should do:

  1. ensure cygwin has the libs libxml2-devel and libxslt-devel installed
  2. issue the command
    cygwin$ pip install pygal

and you’re done. Getting pip into cygwin is a whole world of hurt, but you will need to go looking for a http (not https) source to download setuptools, then download and run ez_setup.py, followed by using pip to upgrade setuptools. Have fun with that; I know I did.

Is Django MVC doing it wrong?

I’ve just starting fooling around with Django (a Python web framework), and was looking to produce a form. Bear in mind that Django doesn’t really do MVC, but follows the philosophy – separation of logic, representation and appearance:

class BookForm(forms.Form):
    title = forms.CharField()

def BookView(request):
    form = BookForm()
    return render_to_response('book.html', {'form': form})

With boot.html containing (amongst other things):

<form action="" method="get">
{{ form.as_table }}
<input type="submit" value="Search" />
</form>

Which is great! MVC, separation of data, presentation and business logic. Now, how do you get a CSS class onto that title field? CSS, being the way of separating out the presentation part of a HTML page from the data that’s embedded in it? As above, but chuck it in as such:

class BookForm(forms.Form):
    title = forms.CharField(
        widget=forms.TextInput(attrs={'class':'title-field'}))

Seeing this crunched the gearbox in my mind. All that messy designer stuff, where they make things look nice, that’s worming it’s way into my business logic? Perhaps it’s not so wrong, as the business logic does indeed know that this is a title-field. But it doesn’t quite sit right with me. I’m not convinced it’s wrong, but if you were, you could instead do this in your CSS and HTML:

<style>
.title-field input {background:#ccC68f;}
</style>
<form action="" method="get">
<table>
<tr><td class="title-field"> {{ form.title }} </td></tr>
</table>
<input type="submit" value="Search" />
</form>

Which pretty much forces you to individually place fields — you get to specify the order of fields plus their individual CSS classes.

I’m not sure what the answer is here. Anyone care to enlighten this noob? Bear in mind that there’s a thing to magically tie a model to a form meaning you don’t even need to specify the fields in both the form and model, which you can’t use if you start tossing styles into each field.