Check SHA or MD5 sums on Windows

I know I should do this more often on my own machine, but I am weak. However, it is a necessity when installing Open Source software at work that I check the checksums after download. I always forget how to do it and often laboriously check the whole thing with MK 1 human eyeball. Dumb.

So, to make it better, here’s the simple, command line way, to check the checksums.

You’ll need:

  1. cygwin installed
  2. cd’ed to the folder where you downloaded the installation file to.

echo "9819c6d21360f805cc17157c64c24672 apache-tomcat-7.0.42.exe" | sha1sum -c -

If it’s good you should get (depending on your gitbash/bash/etc shell you’re using):

apache-tomcat-7.0.42.exe: OK

If bad you get: 

apache-tomcat-7.0.42.exe: FAILED
md5sum: WARNING: 1 computed checksum did NOT match

or maybe

no properly formatted MD5 checksum lines found

Thanks to Les on upthescala

EDIT 20130816: Fixed typo in command to remove pipe and replace with redirect.
EDIT 20140128: What was I thinking on previous edit? Anyway, fixed and added a little more colour.

Advertisements

Serving HTTPS out of Bottle using CherryPy

I wanted to bash a couple of ideas together and see how quickly I could get something serving data from mongodb over https. I’d just started messing with python and the pymongo driver and I was (and still am) loving the quick hit python coding gives. Adding bottle as a framework made it all webby and useful feeling into the bargain :-). What is not not to like?

What I really wanted was a sweet single point solution that bottle already gave me, but with https. Past self takes over at this point….

After a bunch of digging around and reading lots of blurb, the upshot is a lightweight https server called CherryPy using bottle querying a mongodb using a regex and outputting whatever is in the first result to the screen, with a list of some of the other results, allowing the user to refine their search. The app is mickey mouse, and you do have a post where I’d prefer a redirect and using a cookie (so that browser refreshes work nicely), but the amount of data you can potentially return using the re would blow out the cookie size pretty quickly. There’s a probably a better way(tm) 😉

To do a test run you will need:

  • mongod running
  • If you want it not to throw a wobbler (I haven’t put much exception handling in it), you’ll need at least one row in testbottle.trades collection with at least one attribute of ‘Trade_Id’.
    So create a test db and record or two to play with. Start mongo shell and add some:

    use testbottle
    
    //bung one in with an easy one match re to remember
    db.trades.insert(
    	{
    		"Trade_Id" : "000457390G-1",
    		"Version_Number" : 1,
    		"Notes" : "",
    		"Settlement_Consideration" : 1721.72
    	}
    )
    
    //let us bung a small shedload in starting with x
    for (var i = 0; i < 1000; i+=1) { 
    	db.trades.insert(
    		{
    		"Trade_Id" : "x" + i, 
    		Version_Number : i, 
    		Settlement_Consideration : Math.random() * i * 100
    		}
    	)
    };
    			
    		
  • python installed
  • pip install pymongo – OR if on windows – http://pypi.python.org/pypi/pymongo use the windows msi
  • pip install bottle
  • pip install cherrypy
  • pip install pyOpenSSL – OR if on windows – http://pypi.python.org/pypi/pyOpenSSL use the windows msi
  • Create a top level folder for this project. I used pyMongoBottlePoc
  • create your own ssl cert –
    openssl req -new -x509 -keyout testserver.pem -out testserver.pem -days 365 -nodes

    Put this in your project top level folder.

  • The find_trade.py code is here. Put this in your project top level directory
  • The hello page and show found pages are here. Put these in “views” subdirectory
  • Run it by typing at the command line
    python find_trade.py
  • Use it by pointing your web browser at
    https://localhost:8082/

find_trade.py is a webserver on your localhost port 8082 which initially shows a welcome page hello_get_value.tpl asking you to input a trade id. When you click submit, find_trades.py does a regex search in the mongo database returns a cursor (trades) and the first result (trade). Bottle renders them using show_trade.tpl.

I leaned really heavily on this post from Diego XA and on the 10Gen Dev course youtube videos week 1 which have an excellent intro to python course in them. Thanks guys.

Installing Python, pyMongo, and Bottle on Windows

I had no problem installing all of the above on my Mac. On Windows, however, it was a bit of a pain. Thinking about it, it maybe that the mac install was Python 2.7, which everyone has been there and done that, whereas the Windows install was python 3.3, which may be a little later to the party.

Anyway, what I did:

  1. Install python 3.3 from http://www.python.org/download/releases/ Use the msi installer
  2. Install the pymongo using the msi from http://pypi.python.org/pypi/pymongo/ > Look down the bottom to find the windows installers. I suggest you don’t do an easy_install, it won’t compile properly and will either not work at all, or will have degraded performance.
  3. Install distribute. The instructions are here. I found it easiest to install from cygwin/git-bash using:
     $ curl -O http://python-distribute.org/distribute_setup.py
    $ python distribute_setup.py
  4. Install pip using easy_install supplied by distribute. NB: you must do this from the DOS command line (and, it appears, be in the Scripts folder):
     C:\> cd c:\Python33\Scripts\
    C:\> easy_install pip
  5. I also installed bottle at this point.
     C:\> pip install bottle

Many experienced python developers are recommending using virtualenv. It allows you have a virtual python install where you can pip install anything without blowing up your base python install by accident. You may have to use the performance retarded pymongo as Windows 7 (32-bit) doesn’t have the c libraries needed to build the necessary modules.

Hat tip to khattam for the distribute/easy_install usage.

GIT: Find commits for a specific file

git log --stat component/src/main/resources/META-INF/spring/context-osgi.xml

 Gets you a list of all the commits including that file.

git diff $commit_hash_old $commit_hash_current component/src/main/resources/META-INF/spring/context-osgi.xml

Shows you the diffs between the old file and the one you have now.

Cross ref: git blame might be more useful. Tells you which lines of a file were changed by whom:

git blame component/src/main/resources/META-INF/spring/context-osgi.xml

6d4e6fd3 (Dick Dastardly 2012-07-26 15:28:33 +0100 24) VirtualTopic.com.some.co.error.DLC
6d4e6fd3 (Dick Dastardly 2012-07-26 15:28:33 +0100 25) /data/error/
c3cedfe7 (Dick Dastardly 2012-06-13 16:10:19 +0100 26)
6d713a0a (Muttley 2012-10-18 13:44:33 +0100 27)
3233b294 (Peter Perfect 2012-11-07 13:16:34 +0000 28) mappingPersistenceUnit
3233b294 (Peter Perfect 2012-11-07 13:16:34 +0000 29) HSQL
c3cedfe7 (Dick Dastardly 2012-06-13 16:10:19 +0100 30) org.apache.openjpa.jdbc.sql.HSQLDictionary
c3cedfe7 (Dick Dastardly 2012-06-13 16:10:19 +0100 31) org.hsqldb.jdbc.JDBCDriver
47e39616 (Dick Dastardly 2012-06-21 14:12:40 +0100 32) jdbc:hsqldb:data/resource/lookup.db
3233b294 (Peter Perfect 2012-11-07 13:16:34 +0000 35) 1

Windows Batch File Echo empty line

I rarely have to do anything windows batch files, but just sometimes I get the short straw or it’s the quick way around something.

It’s nice to be able to output something reasonably readable to the tracking file, and echoing an empty line can make things easier to parse with-my-eyes. Echo with a following special char (select your own!) does the job.

date /T
time /T
echo doing something…
echo done
echo:

echo. also works. echo? and echo either of the slashes are dubious.

Thanks to macropas on stackoverflow for that one.

MongoDb: Drop a Collection With runCommand

Stupidly I managed to create a collection with a space in the name. In the interactive shell various escape chars and quotes using the “db.CollectionName.drop()” solution failed to work. Since I created the blasted thing with a runCommand, I can fix it with a runCommand:

> db.runCommand( { drop : "my output" } );
{
"nIndexesWas" : 1,
"msg" : "indexes dropped for collection",
"ns" : "preNormalisation.my output",
"ok" : 1
}