emacs rocks

I had a boring text editing task. I'm using a text file that refers to components by a number and after writing it I realised that I need those component numbers to be incrementing.

I had:

Bus.con = [
 101   138  1  0  2  1;
 102   138  1  0  2  1;
...
 124   138  1  0  2  1;
 201   138  1  0  2  1;
 202   138  1  0  2  1;
...
];

Shunt.con = [
 106  100  138  60  0  -1  1;
 206  100  138  60  0  -1  1;
 306  100  138  60  0  -1  1;
];

I wanted:

Bus.con = [
  1  138  1  0  2  1;
  2  138  1  0  2  1;
...
 24   138  1  0  2  1;
 25   138  1  0  2  1;
 26   138  1  0  2  1;
...
];

Shunt.con = [
  6  100  138  60  0  -1  1;
 30  100  138  60  0  -1  1;
 54  100  138  60  0  -1  1;
];

One problem is that the component number might be the same as a parameter (which I don't want to change). Two tools here:

1. re-builder - interactively build a regular expression. 
2. (query)-replace-regexp - replace a regexp with another 
3. elisp in regexps - do anything with a regexp

This led to the following three replacements:

replace " 1\([0-2][0-9]\) " with " \1 "

replace " 2\([0-2][0-9]\) " with " \,(+ 24 string-to-int \1)) "

replace " 3\([0-2][0-9]\) " with " \,(+ 48 string-to-int \1)) "

by doing it as a query I made sure I was only replacing the correct stuff. 

I even had time to write this blog post. 

Setting up a simple GitHub repo

Global setup:

sudo apt-get install git-core
git config --global user.name "James"
git config --global user.email kerspoon@gmail.com

Next steps:

mkdir laos
cd laos
git init
touch README
git add README
git commit -m 'first commit'
git remote add origin git@github.com:kerspoon/laos.git
git push origin master

To Steve Yegge

Below is a good chunk of an e-mail I sent to Steve Yegge. It was basically a summary of what programming related things I have been doing and where he thought it was enough to get a job.

---

First a short list of some of the things I am now doing related to that aim (I'm sure you will recognize some of these from your posts):

+ I subscribe to a number of programming blogs
+ I use emacs (though I'm still not happy with my configuration)
+ I started learning to touch type (in Colemak)
+ I'm programming in lots of languages
+ I'm trying to optimize the way I work
+ I read a bit of wikipedia, stackoverflow, ted.com most days
+ I've started Project Euler challenges in python
+ I have written a minimal Scheme interpreter in C++ & Digital Mars D
+ I have written very simple scripts in BASH/grep/find/cat/sed
+ I use Linux (in the lab), Windows (at home) and Mac OS (at work)
+ I have read SICP, GEB, Design Patterns, The Pragmatic Programmer

About me (if you're actually interested)
========

I am a UK PhD student currently studying Electrical Power Systems, which involves a lot of work on artificial intelligence. Before that I did a BEng in Electrical Engineering involving writing a Genetic Algorithm to optimize settings on a controller for a virtual mag-lev train. I wrote the GA, the integration algorithm, the simulator and the GUI in C++.

Started programming in Macromedia Director Lingo, moved on to VB/VBA in school, taught myself HTML, CSS and a little bit of PHP so I could design a couple of websites. Took an online course in Game Programming that taught me C++ & DirectX/Win32 API. At University I had courses on Unix & C, Java, Matlab, and PIC programming in assembly. I would only really say I am comfortable writing C++ and python, I can read and understand the others, not sure why but I really dislike Java. I haven't been taught much computer science but I think my math is good.

My Aims
========

+ Learn a pure functional language
+ Learn a bit of Forth
+ Learn a more of Python
+ Read a book on:
- Compliers
- Algorithms
- Regular Expressions
- Mathematics for Programmers
+ Read the following:
- Programming Language Pragmatics by Michael L. Scott
- Hacker's Delight by Henry S. Warren
- Refactoring by Martin Fowler
+ Improve my scheme interpreter by:
- Implementing macros
- Implementing call/cc (then hopefully understand it better)
- Re-writing in python
+ Learn how to use threads

The Questions
========

1. I'm struggling to learn a different keyboard layout, do you think it's really worth it?

2. As I haven't done a CS degree is there any thing I really need to know that I haven't covered?

3. Out of my aims which do you think are more important, are there any pointless ones, or ones you would like to add?

4. Do you think that if I achieved my aims I would be good enough to be hired?

5. Is there resources other than Project Euler that will give me tasks to make me learn a language better.

6. How important is doing the exercises in the book I read? How many do you do when you read a new book?

My emacs Annoyance
========

I found emacs to be like my iPhone; amazing yet annoying. When I got my iPhone I was amazed at all the new things I could do and how easy most tasks were, shortly after that I was annoyed that they hadn't done more, it's not complete.

emacs has the potential to be just about the best application ever made but for me the default settings suck. It took days to be able to use (which isn't a problem) but weeks in and I'm still struggling to get things like auto-completion working how I want it to. It's as if you have to live in emacs and hack everything together yourself. Still, I cant go back to any other editor now, emacs is just too good. But I will remain frustrated that all those things that I want to do and I know are possible are going to elude me, unless I take the time to get it working myself.

It's like the difference between Firefox and Opera. Firefox gives you a pretty basic web browser with the magic ability to add to it. Opera gives you an advanced web browser out of the box without the ability to add to it. I used to use Opera because the addons for Firefox don't work well with each other and it would take a long time to get it working how I wanted. It's the same with emacs, but there is no equivalent (maybe TextMate).


Before this get too long I will end it, thanks for reading up to this point.

James Brooks

Mocking Python Files

Just a quick note on testing python; I needed to test a method the reads in from a file. To do that I could either created a temp file or use stringio. Stringio is easier and quicker so I used that, here's how:

def mockfile(text):
    return StringIO.StringIO(text)

class Tester1(unittest.TestCase):
    def setUp(self):
        self.filehandle = mockfile("""
        gen , G1  , 101 , 450  ,  50  ,   U20""")
        self.components = read(self.filehandle)

    def tearDown(self):
        self.filehandle.close()
        self.filehandle = None
        self.components = None

    def test_name(self):
        self.assertEqual(set(self.components.keys()), 
                         set("G1"))


class Tester2(unittest.TestCase):
    def test_a(self):
        components = read(mockfile(""))
        self.assertEqual(len(components),0)

Hopefully I this is the first in a series of posts on practical TDD. Watch this space

Test Driven Development (pros and cons)

I am a fan of TDD (Test Driven Development), for those who don't know TDD here is the places to read about it:

Test Driven Development by Kent Beck

The Three Rules Of TDD by UncleBob

1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
   

Three Monkeys of Test-Driven Development by Ashutosh Nilkanth



You basically make sure everything works before doing anything else; and you only do one thing at a time. It has the great advantage that you can change whatever you want in your code and you can be sure it is as good as before. This allows you to aggressively refactor. Being able to refactor without fear of breaking something should leave you code easier to maintain and easier to add new features. This will hopefully overcome the time it has taken to write all those tests in the first place.

A few things make TDD more difficult:

• Randomness (Non-Deterministic Methods)
  
• User Interfaces
• External Code (including Databases)

It is for this reason that I find it difficult to use TDD for all my coding. Now those who support it say that you can mock up facade classes for the external code including random number generation meaning you can test only the bits you want.

My other problem it it seems to break my flow of thinking. I want to type out a load of different ways of doing things and see which one I like better.

---

I think TDD has it's place but there is a need to do another kind of programming. Exploratory programming; one where you do not know what you want to end up with. You just chuck out a load of code and see how it runs. This is great if you have a nice fast development cycle, i.e. interpreted high level language. I would say that it get exponentially more complicated to do exploratory programming as the total number of lines of code increases. This gives a natural limit to the kind of things that can be accomplished with this technique. I believe it is best suited for tracers and prototypes (see The Pragmatic Programmer). In this case you quickly see the shape of a small part of your program and begin to see what things (modules, classes) you need in the full version. Once you have got a section of code working you can write it properly (with TDD) meaning you live by the rules of TDD and throw one away (Mythical Man Month).

Genetic Algorithms in Python

Following on from before here is some resources for programming GA in python. The top one looks like the easiest to understand and hence is my preferred one.

• http://code.activestate.com/recipes/199121/
• http://www.alextreme.org/projects/python_ai/
• http://pygp.sourceforge.net/
• http://www.freenet.org.nz/python/pygene/

As a side-note here is a list of neural network resources in Python:
http://www.answermysearches.com/four-free-neural-network-libraries-for-python/195/

I'm thinking or reviving one of my old projects; to recreate critters from aijunkie. I have quite a few ideas I would like to add to them once it's up and running.

Python Date and Time

Same project, different problem. This time the problem is dates and times. Specifically converting between them. These two small functions do just that.

# parse_date :: Date -> String
# e.g. parse_date("2001-12-31")
def parse_date(date):
 return datetime.date(*strptime(date, "%Y-%m-%d")[0:3])

# str_date :: String -> Date
def str_date(date):
 return date.strftime("%Y-%m-%d")

• http://docs.python.org/3.0/library/datetime.html#strftime-behavior
• http://docs.python.org/library/time.html
  
• http://pleac.sourceforge.net/pleac_python/datesandtimes.html
• google search
  

P.S. Pleac is a wonderful site, it aims to have a load of common code for a variety of languages, good to get practical tips as well as learn a new language.

Combining Python Dictionaries

I had a small task where I had to take two existing dict and combine them by adding together the values of any elements that already exist. Here's how I did it.

def dict_combine(dict1, dict2, combiner):
      """takes two dictionaries and combines them
      such that the value is the result of:
      result[a]  = combiner(dict1[a],dict2[a])"""
      result = {}
      dict1_keys = set(dict1.keys())
      dict2_keys = set(dict2.keys())

      # keys uniuqe to dict1 are included unchanged
      for x in dict1_keys - dict2_keys:
            result[x] = dict1[x]

      # keys uniuqe to dict2 are included unchanged
      for x in dict2_keys - dict1_keys:
            result[x] = dict2[x]

      # keys in both dictionaries get combined then included
      for x in dict1_keys & dict2_keys:
            result[x] = combiner(dict1[x],dict2[x])

      return result

def test_dict_combine():
      dict1 = dict(a=1, b=2, c=3)
      def adder(a,b):
            return a + b 
      assert dict_combine(dict1,dict1,adder) == dict(a=2, b=4, c=6)
test_dict_combine()