Journal
-
#
The key to being successful at anything, in particular the key to motivation, is to care.
What exactly is it that I care about?
-
# Today, we are all Australians
I know you were expecting to see this here and I didn’t want to disappoint.
-
# Some notes on my Threading model
In the last post I mentioned a threading model, and how I had forgotten how it worked. And having more recently discovered that others have also developed the same model and even given it a name, but that I have forgotten where I read about it.
With some small hope that someone will recognise it and remind me what it is called, I present
A Threading model
Posit the following set of nodes
The nodes are stored in the following tables:
nodes
id parent_id title 1 NULL A Post 2 1 First! 3 2 Lame! 4 2 re: First! 5 1 re: A Post 6 1 re: A Post_2 7 6 re: A Post_3 threads
root_id node_id 1 2 2 3 1 3 2 4 1 4 1 5 1 6 6 7 1 7 root_id is the id of a node, node_id is the id of any descendants.
For example, Node 3 (Lame!) is a descendant of nodes 2 (First!) and 1 (A Post), so two records appear, one with root_id 1 and another with root_id 2.
To select the descendants of a root node:
select n.* from nodes n, threads t where n.id = t.node_id and t.root_id = @root_id
If we were then to add an 8th node, and make it a child of node 3, the nodes table would become
id parent_id title 1 NULL A Post 2 1 First! 3 2 Lame! 4 2 re: First! 5 1 re: A Post 6 1 re: A Post_2 7 6 re: A Post_3 8 3 re: Lame! and the threads table would become:
root_id node_id 1 2 2 3 1 3 2 4 1 4 1 5 1 6 6 7 1 7 3 8 2 8 1 8 the query for updating the threading table being:
insert into threads (root_id, node_id) select @parent_id, @node_id union select root_id, @node_id from threads where node_id = @parent_id
The reason I came up with this system1 was because I didn’t like having to recursively call back to the database for every node to see if it had any child nodes. I was working in a bubble and I’m not sure that I would have found information on Nested Sets If I had searched. Even if I had, I’ve always had a problem with the idea that an INSERT requires followup UPDATEs to the same table, with potential locking problems, &c.
A slight variant that would probably make Database Normalisers cry
I actually used to use a slightly different model back when I wasn’t using ActiveRecord to abstract away queries. ActiveRecord insists that I use foreign keys – even if in the (MySQL) database they aren’t actual formally declared as such. When I originally designed this model, I didn’t worry so much about it and did this:
root_id parent_node_id 1 2 1 6 and
select distinct n.* from nodes n, threads t where (n.parent_id = t.parent_node_id and t.root_id = 2) union select * from nodes where parent_id = 2
Again, if I added an eigth node as a child of Node 3, the new threads table would be:
root_id parent_node_id 1 2 1 6 2 3 1 3 and the query to perform that update would be
insert into threads(root_id, parent_node_id) select root_id, @parent_id from threads t, nodes n where t.parent_node_id = n.parent_id and n.id = @parent_id
I have never done any performance testing on either of these two models, versus Nested Set or even each other. I don’t really need to, I’m not getting enough traffic to need any optimisation on that level. OTOH, I’m not getting enough traffic to warrant coming up with this boondoggle just to avoid multiple database hits to display a comment thread. But it was there.
1 Independently came up with. I know that it’s been developed elsewhere and long before I thought of it.
-
# Some navel-gazing
The software I use to blog with has always been custom written by me, from scratch. I started off with some PHP scripts I slammed out in an afternoon and deployed that very night. I added to those scripts for awhile, and then let it lie fallow for a couple of years. When I started playing with Ruby I started and threw away a couple of ambitious redesigns before Rails came along and I built the current software. Which I have then left to lie fallow for a couple of years. At the moment there is a another version that I have been fitfully working on every few weeks for the last year. It may well be left to lie fallow for a few years without ever being deployed.
I have this tendency to write just enough to do what I need to do, even if the “just enough” involves some very rough edges that need to be stepped carefully around. The first cut of the PHP software acknowledged this by having a script that allowed me to dump SQL statements into a text box and then execute them against the database. This being before I was on a host with phpmyadmin, and before I had bothered to write an admin interface. Before, for that matter, I had even bothered to have anything hiding behind a username and password. Ahh, memories. I also remember at one stage with the PHP code whenever a comment came in I had to manually add it to the threading because I had broken the code for doing it and had forgotten how my threading worked1.
It’s not that I am incompetent or dangerously lax. I do cross all these Ts, dot these Is when I am performing work for my employer. But when writing software for myself, I want to move onto the fruits of my labours. I was the same way when I was a kid and trying to DM, I was much more interested in the act than the process, I would always leap to playing the game and forget about doing the prep work, leading to some games with a suck rating that would have interested Hawking.
So why do I write my own code if I’m not deriving such a level of enjoyment from the process of creating the code that I am encouraged to push on through and do a complete job? Well, I am employed as a coder, so there is a not insignificant element of not being able to face more of the same when I get home from work. There’s also elements of being stubborn, that I should be able to, analogously, fix my own car. And a large element of Because It Is There, that irrational need to do something because you can.
Because any rational examination of the matter would lead me to install Movable Type. It’s madness that I should be solving – or rather not solving – problems that are Movable Type’s core business. Cross Site Scripting vulnerabilities? Fixing them is a high priority for them. Not having urls fall over because of some non-ASCII characters fell prey to a URI Decode? I reckon MT might be all over that. The list can go on and on; it makes no sense for me to try and maintain my own software when there are well-resourced groups to whom I can outsource the effort.
So? Why should that worry me? I’m under no delusions that this blog will ever be read by anyone other than my friends and acquaintances. I’m nobody, I can afford to run a lax process for my own stuff, because it doesn’t matter to anyone but me.
It is quantum of self-discovery I made while I was trying to fall asleep. I hope it allows me moments of comfort in the middle of spells of self-doubt.
1 It’s not a nested set, there’s a separate table that indexes each sub tree, so I can extract with a single query all nodes that are descended from any arbitrary node. I stumbled across a paper discussing the concept, but I can’t find the link any more.
-
# A new job
I resigned from my current job yesterday. My last day will be September 25th, then I start work in a new role on Monday the 28th.
I am pretty sure I didn’t piss and moan about this at the time, least ways not on this venue, so for completeness, I was informed in May that my employer intended to renew my contract but only for nine days a fortnight, leaving me with a 10% paycut. At that point all the careful balancing of pros and cons of leaving fell apart like a madly misplayed game of jenga; I was leaving, no matter what. It took me three months. When I started out a recruiter told me ten weeks was the average time it was taking people he worked with to get a new job. Given I was overly picky and pretty relaxed about it all – after all, I wasn’t unemployed, just underemployed – 13 or so weeks is probably on par.
My current employer is a travel company. They sell packaged tours. One of their tour brands is like a Kontiki for sad, wet people. Another sells river cruises for people who suffer under the burden of having too much money. Most of the rest is bus tours for pensioners.
I don’t know if you’ve heard, but apparently the world is undergoing some sort of massive correction in the financial markets. You might be able to imagine what that kind of thing does to travel companies. Suddenly everyone feels like they should hold onto their money. To cut a long story short, it means that my employer had to cut costs.
Allow me a few sentences of rant. I probably won’t get to say this in an exit interview, so I’m saying it here.
The executives all took pay cuts; contractors were required to take cuts to hours. I understand this, they had a problem and they needed to solve it. However the solution to their problem was, partly, to make a small part of their problem a large problem for me. My solution to my problem then is to return their problem to them. With interest. “You want me to cut my hours? How about zero? Does zero work for you?”
I am the only person who has ever worked on this project, this project that handles a significant chunk of their daily business processes. Not the bits that directly make money, but the marketing bits, the bits that lead up to some of that money being made. It doesn’t matter how good my documentation and hand over are because whoever gets lumped with responsibility won’t have four years of working with this thing behind them. The accountants who decided to cut my hours may well find that however much they saved by only contracting me for nine days instead of ten will get blown away by the time wasted having someone else re-learn all the subtleties, knowledge that walks out the door with me.
Rant over.
My new employer is a communications agency. But effectively for me they are a web development agency. I’m looking forward to going to web development. It was what I did primarily until about four years ago – when I became more of a database and middle-ware developer – and I’m excited to be going back. With the usual trepidation. While I’ve tried to stay on top of new developments, I’ve been a little left behind. The next three weeks are going to be interesting while I brush up on everything I should know. But it is a fresh start. I no longer have to support decisions I made as long as four years ago, decisions that are painfully naive in light of things I have learned in since making them. I get to make all new poorly informed decisions, and I get to make them as part of a team.
-
# la résolution des problèmes de l'escalier
I was asked today to write a function to calculate the number of bits set in a given integer. That is, there is one bit set in 4 (100), but two bits set in 5 (101) .
int CountOfBitsSet(int n) { int count = 0; while (n > 0) { if (n%2 == 1) count++; n = n >> 1; } return count; }The follow up question was the maximum number of times the main loop would..err…loop1. The follow up to that was how I could optimise it. I didn’t know, and it was suggested to me that you could use a lookup table. I agreed that if it was something that was likely to remain in memory for awhile and be used a lot, you could maintain a hash of previously calculated values, but what they were really getting at was a lookup table that is preloaded. That’s a lot of memory, that is for just over 2.1 billion numbers.
Obviously the last follow up question was how you could optimise this ridiculous unhelpful lookup table. I couldn’t think of one, I was too distracted by how ridiculously unhelpful that lookup table would be and just suggested that I wouldn’t do that, I’d just use a populated-as-generated hash.
There’s a term for thinking of the perfect comeback ten minutes too late; is there one for thinking of a good answer to a question a few hours later2?
It’s in Ruby instead of the C# I was being tested in. Since they weren’t testing me in the language so much as problem solving, I’m quite sure it wouldn’t have mattered.
class BitCounting LOOKUPS = {"0"=>0, "1"=>1, "2"=>1, "3"=>2, "4"=>1, "5"=>2, "6"=>2, "7"=>3, "8"=>1, "9"=>2, "a"=>2, "b"=>3, "c"=>2, "d"=>3, "e"=>3, "f"=>4} def self.numBitsSet(n) hex_ver = n.to_s(base=16) count = 0 (0..hex_ver.length-1).each{|digit|count += LOOKUPS[hex_ver[digit]]} count end endMaximum number of iterations through the loop drops to 8, and the lookup table has a negligible footprint.
Bah!
1 31.
2 Apart from too bloody late by half.
-
# Jury duty
In August of 2007 a security guard for a company that provided security for ATM technicians finished his shift early, went through all the procedures for signing in his weapon and keys, left the premises and then allegedly broke and entered an ATM bunker and stole $270K+. The next morning he turned up for work and hour and a half before his shift started, appeared to break certain parts of the procedure for being assigned keys, including taking keys out of the secured area, out of shot of CCTV cameras, before they are audited. Four days later, after a period during the police couldn’t contact him, he was lured into work by his boss with the pretense of a team meeting, and promptly arrested by NSW police detectives from the Liverpool command. He was taken to the station and then proceeded to give a hour-long statement in which he appeared to constantly repeat a number of rehearsed lines. Or maybe he was just repeating the only explicable things, the security procedure he followed, specifically the bit that if it had indeed been followed, would exonerate him.
I know this because I spent the last fortnight as part of the jury. I now know more about the security of ATMs as operated by Linfox Armaguard then most people. Possibly more even than some of the people actually responsible. I know that the electronic security system for the locks has never been tested properly, relying instead on security by obscurity. I know that the system of keys, commercially available iButton devices easily obtainable from the internet. The emulation of these is a project routinely assigned to first year Electrical Engineering students, in Sydney and elsewhere; a fact attested to by a man who holds a Top Secret clearance in Australia, does security consultation work for the Department of Defense and ASIO, is currently doing testing work for international air traffic control – with the aim of simulating bringing down an airplane – a course adviser for the Electrical Engineering department at one of those Sydney universities, a man who in the act of pwning the expert witness from the firm that supplied the security system mentioned that he personally had conducted white hat operations against a security system functionally indistinguishable from that used by the ATM.
I know that the police can and do get a species of target fixation. They’ll focus so much on who they like for the crime that they forget to actually make sure of everything, to cross the T’s and dot the I’s. They’ll not think to go back over footage from the weeks prior to the crime and check the accused’s assertion that all the guards take the keys off camera and into the kitchen prior to the filmed auditing process. They’ll not bother to take a statement from the technician who was at the ATM the day after the robbery and, through confusion at the bank’s Helpdesk, did not realise that the reason the ATM was empty was not because of an official decashing, but a very unofficial one. They won’t take this statement until two years later, on the penultimate day of the trial for the crime. They won’t go to the operators of the security system and thoroughly and completely trash out what has happened, a process that might have eliminated one of the alternative narratives raised by the defense – that of a rogue security technician – and would almost certainly have prevented a security technician having to tell the lawyers during a visit to the crime scene that he can’t explain why the safe even opened since it was in the lock out period when the keys can’t even open the damn thing, only to have to back down the next day when he finds out he is mistaken about when the lock out period starts each night. The Crown prosecutor, for at least twelve hours, thought his entire case had been blown out of the water. This occurred the day after the jury was empaneled and the trial had begun. Brilliant.
As you can probably already tell, we the jury found the accused not guilty. Right up until the last hour or so of the defense summation there was still doubt which way it would go. I had been stressing over the decision, unsure which way to go: I was reasonably sure he had indeed done the crime but I knew I was basing that on some inferences and didn’t want to send a guy to gaol when he might not have done it. The stress was stupid, I knew it was stupid at the time, I knew that if I wasn’t sure I had to go with not guilty, but the knowledge just couldn’t penetrate the outer shell of “I am pretty sure he did it”. But then defense mentions during his summation that some of the inferences I was drawing weren’t just invalid in a jury decision, but completely invalid in reality.
The trial took nine days. Deliberations took twenty minutes. Could have been five, but one of the jury members deliberately voted against the flow so that he could make sure we would still get lunch for the day. Well, also because he thought five minutes might have been a bit unseemly. But the lunch bit also. I had been sure there was going to be hours of circular arguments because half of the jury had spent the entire trial going over irrelevancies in the evidence. In the end though it came down to whether the accused had access to three distinct things. He certainly had access to one, the electronic keys, and everyone else believed he had access to a second. I didn’t and while I stood my ground in a discussion on it, it didn’t matter because nobody in the room could say that they were sure beyond reasonable doubt that he had the third item. Yes, it sounds like Cluedo; The security guard, in the ATM bunker, with the e keys, the keys and the alarm code.
The forewoman – all of twenty-four – was sweating bullets about having to read the verdict. She thought she had to speak directly to the accused. She thought she had to explain why we made the decision we did. No, she just had to say two words to His Honour’s assistant, sit down and then within seconds we were dismissed and away we go. Hoping never to see the accused again, a hope that might be forlorn in the case of the forewoman, because as it turns out, she lives two streets away from him. Even if she does see him, it has to be better than bumping into him on the street as happened for a number of us. In cafes while getting coffee in the morning. You’d walk in, see him and walk back out again because it’s just too awkward to bear.
That’s how I spent the last two weeks. That doesn’t explain the preceding seven weeks, but that time is lost.
-
# Today, we are all stateless
-
# Chicken Soup for the Belly
I made chicken soup this week. For want of anything better to blog about, this is how I made my chicken soup.
- Go buy a big honking1 chicken.
- Roast it2.
- Eat the chicken, or at least the meat bits. Probably also the skin. If there’s any meat left on the chicken, remove it and save it for chicken sandwiches.
- Put the rest of the carcass in a pot of water with a chopped carrot, chopped celery stalks, salt, pepper and enough water to cover it all. Bring to boil then let it fester and bubble for…ehh…awhile. I let it simmer for a few hours, turned it off overnight, then started it simmering again in the morning and then had it turned off again at lunch-ish.
- Remove the carcass. Probably with a slotted spoon. Dispose of it. Thoughtfully.
- Add chopped parsley and thyme – about a tablespoon of each – and an ear of corn chopped into sections. Bring to the boil again, then let it simmer for a few hours.
- Add some macaroni maybe 60-90 minutes before you serve.
- Serve.
I’ll take a picture next time. Or you know…make it, then you’ll have a picture made of chicken soup.
Meanwhile as I type this, there is a show on Foxtel that appears to be ten to fifteen minute long segments of a strict format: “Could x happen3? Followers of the Seven Signs prophecy say yes!”
1 Note, in this case honking means big. Not actually honking. Not least because that would mean you’ve got a live bird and we live in the 21st century, you can pay other people to kill and clean your meat products. Also, that would be a goose.
2 Roasting left as an exercise for the reader.
3 Where x is something so unlikely to occur that you’d have to be an absolute fucking moron – specifically reared to be as gullible as all get out – to even consider the possibility that x could occur.
-
# Several days ago, we were all Argentinian...and Dutch
I am disgusted with myself, simply appalled at how out of it I am, the depths into head and my navel that I have sunk to and led me to not be aware how much I needed my country to have territorial dominion over a cold, windswept shit hole.
Still, I do, and shall forever more, just like to see them lose.
