Monday, August 27, 2007

Graham, On Point

{

I was just missing the old Paul Graham, the one who would write wisdom for the type of programmer that I imagined myself to be. The most recent essay, Holding A Program In One's Head, is that treat I've been waiting for.

}

.NET 301 Redirect

{

Scott Hanselman was noting that many people who'd updated the locations of their tools on his 2007 Ultimate Developers Tools List had emailed or contacted him asking him to update his reference.

He lamented the fact that they didn't simply use HTTP 301 redirects (Moved Permanently) which tools can easily use to update referenced links.

I must confess, I was one of those people.  I just lazily made a default.asp that had one line:

Response.Redirect("http://www.nregex.com")

But 301 redirect are pretty simple whether you use .NET or lazily throw an ASP script somewhere.

'classic ASP
Response.Status="301 Moved Permanently"
Response.AddHeader "Location", "http://foo/bar.asp"

'.NET

Response.Status = "301 Moved Permanently";
Response.AddHeader("Location","http://foo/bar.asp");

Source: seoconsultants.

}

Friday, August 24, 2007

NRegex

{

I embarrassed myself today (yelling at the work desk, telling everyone I'd take them to McDonalds) but with good cause: a tool I wrote some time back made it to Scott Hanselman's 2007 Ultimate Developer/Power User Tools List. The goal of the tool was simple: to do browser based evaluation of regular expressions using the .NET engine.  Prior to writing it I would frequent Rexv.org but because the engines supported there were different, I'd usually have to do some conversion to get things to the .NET world.

There are quite a few .NET Regular Expression tools: Expresso, which is my first and favorite since I've used it longest, but also tools like Roy Osherove's The Regulator. The problem with these tools, however, was that as fat clients I wasn't always easily able to use them: I work on different machines (home, work, client sites) and also assist people quite a bit away from my desk.

There are also quite a few Regular Expression tools for the web, Rexv, which I mentioned before but also RegexPal. But these tools support either javascript or alternate implementations so patterns like (?<foo>\w+) usually return some kind of error. 

The way Nregex works is pretty straight forward: I'm using YUI for the Ajax library and working with the .NET implementation of the regular expression on the server.  A simultaneous goal of mine is to get into more of YUI because I'm always building little knick-knacks in javascript and I think it's the library to beat right now.

Of course it's a fledgling: I worked on it a few nights when I could and fixed bugs here are there when I had a moment.  Please use it and give me feedback since that will be my biggest help in making it better.

One more thing: I'm trying to collect useful regular expressions. I'd like to build a good library of expressions to help users of the tool. Use the feedback form and let me know if you've got one you've found handy. Don't worry if it's not .NET specific, I'll do the translation.

}

Sunday, August 12, 2007

What Comes First?

{

Although it's an older essay (November, 2004), The IDE Divide made for some good Sunday reading. Not only is the author, Oliver Steele, engaging but his visualizations are exceptional. I'd like to think of myself as a "language maven" - that is a person who puts the emphasis on language capabilities before tools - but I know that growing up in the Microsoft world as I did, I understand how big of a difference good tools can be.

One item I don't recall seeing in the essay was how tool mavens are responsible for their own obsolesence. It makes perfect sense to focus on language first and be flexible with tools - and to have a good text editor you can rely upon. I started using Textpad first with Java and nowadays I usually don't pass a day when I don't open something with it - either for quick inspection of a file or to write something in perl or python.

I know there are other good text editors out there but it seems like having good regex support and the ability to attach shell commands to keyboard shortcuts is most important. Syntax highlighting is a nicety too although as time passes it's not really as important to me.

}

Tuesday, August 07, 2007

!Normalize

{

Some time ago Jason Kottke posted an entry for which quite a few people bashed him about unnecessary database normalization. I've thought a lot about it since it represents some of my own internal sentiments and instincts. The two things I observe on a regular basis that I find annoying are bad naming conventions within databases and unnecessary complexity because of overzealous normalization.

It's hard to have an argument discussion on the topic with people lacking experience because the penalties for bad normalization are usually paid in the long term whereas the "effeciency" of storage with normalized data seems to be an ideal to strive for when one's perspective is short.

I was therefore heartened to see some caution from Patt Heland (courtesy of Dare Obasanjo who elaborates) on the topic. It may be a good idea to have a meeting at work and discuss some of the details given... maybe not to come to some "conclusion" or "database standard" that can apply to every situation, but more to generate good discussion and ideas.

}

Wednesday, August 01, 2007

Upsert Properly

{

For the longest time, my approach to update/insert logic has been the following:

IF EXISTS(SELECT...) BEGIN
UPDATE...
END
ELSE BEGIN
INSERT...
END

So courtesy of .NET Kicks this gem was very informative which is the same logic but with a cleaner approach. Instead of selecting records first, you attempt to do an update and check afterwards if the @@rowcount is greater than zero - if not the record doesn't exist so you can move logically to an insert.

UPDATE ...

IF @@rowcount = 0 BEGIN
INSERT...
END

Nifty, very nifty.

}

Tuesday, July 31, 2007

Fast Learning, Norvigian Thinking

{

I was looking over Justice Gray's post on becoming a better developer in 6 months and noticed that he had listed Freidl's Mastering Regular Expressions as a one week reading project. I'll preface my comments by admitting I am not the quickest read - unless I'm reading something I don't particularly care about I tend to go it slow no matter what the subject matter.

However.

Not really thinking of connecting the dots I checked in on Jeff Freidl's blog and digging around saw it took him 2.5 years to complete the first edition of the book.

It strikes me as funny that there would be such a disparity in creation and consumption. Of course I may be slow - a big part of my wanting to learn Perl was the desire to have the capacity to "think" in regular expressions - but over the last few years as I've gotten better and better I find it hard to think I could compensate the little projects and tools I've written without the pain and grit of using what I read slowly.

But then again we can all rest assured that there's no rush.

}

Sunday, July 29, 2007

Remove Duplicate Lines In Python

{

I had posted about the set operator in Python with some questions. All that changed today when I wrote a little script to remove duplicate lines from a file. The set operator takes a list and automatically gets rid of duplicate items. Very useful for situations like this:


#!/usr/bin/env python

f = open("c:\\temp\\Original.txt")
f2 = open("c:\\temp\\Unique.txt", "w")
uniquelines = set(f.read().split("\n"))
f2.write("".join([line + "\n" for line in uniquelines]))
f2.close()


}

Saturday, July 21, 2007

Shuffled Arrays

{

One of my weaknesses is that I love puzzles. And once I'm puzzle solving, I usually dwell on the problem beyond its worth. I recently saw a job ad - I'll leave this post disconnected - that had a quiz associated with it. The quiz amounts, basically, to shuffling items in an array (javascript).

My first stab was intuitive, but I wonder if it's the most optimal because it relies on a lot of discarded data. In my loop generating a random order, I essentially go through an undetermined amount of times discarding results that already exist in the randomized array. Additionally the array with random numbers is just for positioning and is probably unnecessary.
Here is the code:


function BuildArray(){
// just builds a random array to work with
var testArray = new Array();
testArray.push('Test 0');
testArray.push('Test 1');
testArray.push('Test 2');
testArray.push('Test 3');
return testArray;
}

function GetShiftOrder(testObject){
// this is what sorts things out
var upperBound = testObject.length;
var newOrder = new Array();
var shuffledArray = new Array();
builder:
while(newOrder.length < upperBound){
n = parseInt(Math.random() * upperBound);
for(i=0;i<newOrder.length;i++){
if(newOrder[i] == n){
continue builder;
}
}
newOrder.push(n);
shuffledArray.push(testObject[n]);
}
alert(newOrder + '\n' + shuffledArray);
}


I was thinking about this today and it may be more effecient to make a random number of passes at the array swapping pairs of items. A shuffling cards approach may be slightly more effecient although to have meaningful swaps there would need to be a minimum number of passes - and additional complexity with making sure pairs were swapped in random pattern.

Anyway, it was interesting and I'm always curious about more elegant solutions.

}

Thursday, July 19, 2007

CodePress

{

I'm taking a serious look at CodePress. Very, very, very cool.

}

Sunday, July 15, 2007

Sieve of Eratosthenes, Python, Set Operations

{

I pretty much broke down after TechEd. I thought I'd be patient enough to wait for Ruby but because Python is most mature I have started to learn it, whitespaces or no.

The first thing I did was check out Guido van Rossum's tutorial for programmers, which was an excellent first step. I've followed that up with some random programming - a lot of fun so far.

I was wondering about the set operations in python and how that made a difference in programming since there aren't syntactical equivalents in .NET. Incidentally at the time I was reading The Man Who Loved Only Numbers and ran across a short description of the Sieve of Eratosthenes as a way of finding primes. I thought it would be a good way to check out the set operations of Python.

I wrote the following:


from System import *

def multiples(num, thresh):
multi = []
for i in range(2,thresh):
m = num * i
if(m < thresh):
multi.append(m)
else:
break
return multi

primeThresh = 5000
print DateTime.Now.ToString("MM/dd/yyyy hh:mm:ss")
nums = range(2,primeThresh)

for n in nums:
nums = set(nums) - set(multiples(n, primeThresh))

print nums

print DateTime.Now.ToString("MM/dd/yyyy hh:mm:ss")


Interesting, but slow. The way I may do something like that in C# (which also works just fine in Python) would be this, which I wrote later:


from System import *

primeThresh = 5000
print DateTime.Now.ToString("MM/dd/yyyy hh:mm:ss")
nums = range(2,primeThresh)

for n in nums:
for m in range(2,primeThresh):
try:
mult = n * m
if(mult > primeThresh):
break
nums.remove(mult)
except:
pass

print nums

print DateTime.Now.ToString("MM/dd/yyyy hh:mm:ss")


It's a lot faster than the previous approach which makes sense - doing set algebra on large sets should take a long time... but that begs the question: are those set operations dangerous (ie. so slow as to be costly).

I'm wondering what's "pythonic" and how a jedi would write this most effeciently.

}

Saturday, July 14, 2007

Being Better

{

I probably won't be officially "tagged" but a meme is going around about what one would do in the next 6 months to become better. Hanselman, in his podcast, spoke of a few things and the responses seem to be going up around the blogs.

How will I be better? Most of the things people mentioned are things I already try to do: reading technical books, working on my software, looking at open source, training others... but one thing I want to do is to start posting code.

I write a lot of code and much of it doesn't make it here because I'm sheepish about looking foolish. But getting better is about having courage to show my work because without that, how could it get any better?

I'm going to start with my JAPH, a program that prints "Just Another Perl Hacker." It's meant to be novel and I actually like mine since it leverages some language features that don't exist in my mainstay, C#. The map operation can visit each item on an array performing some opertion - in this case I'm using a regular expression on each item and grabbing alternating characters! I haven't been writing perl for a long time so hopefully over time I can claim a better one but at least this is my own thinking at play, destined for improvement. I did post it on usenet (a first version which had some foolish mistakes) but this is after a little bit of massage:

#!/usr/bin/perl -w
# David's JAPH, volume 0.1
map(print(/(\w)\w/g," "),
qw(Jaubsctd Aenfogthhiejrk Plemrmln Hoapcqkresrt));


The original JAPHs, from Randall Schwartz, were not really about obfuscation but more about language features. The use of map, regular expressions, and $_ in the above are what make it interesting to me.

}

JLam on DNR

{

I never mind a long drive with a podcast. A few days ago I listened to John Lam on Dot Net Rocks. I'll admit that sometimes the DNR people can be annoying to me (think Richard Campbell saying over and over like it's a joke "Managed JavaScript??") but it was all worth it to hear some of the internal goings on with the Iron Ruby project.

Gleaning: Iron Ruby is a ways off. I got somewhat impatient and have been using IronPython in the meantime, but more on that later. That's not a small decision for me because I am of the Norvig perspective on learning a new language - it takes the length of time needed to think in terms of the idioms of that language, as opposed to just writing your code with different syntax and keywords (ala writing C# in Ruby).

Gleaning: Ruby is a powerful language for creating domain specific languages. That meme has been floating in my head for a while - listened to a good Software Engineering Radio podcast on the topic sometime back.

Gleaning: JLam talks a bit about how the learning curve for new languages these days revolves around the frameworks for languages - I think the idea of Ruby combined with .NET is that the .NET libraries will fill a gap that Ruby has always missed in library support. Dangerous or brilliant? In other words, I'm sure the open source crowd (is/will be) up in arms because this will, in their minds, dilute the community effort behind a language. Brilliant because the Mort you know (who is probably your boss) will have a better level of confidence about using it.

Gleaning: Iron Ruby is implemented in C# but Managed Javascript for Silverlight is a VB.NET implementation!

}

Monday, July 02, 2007

True Story: The Cat Ate My Source

{

A security issue prompted my ISP to change passwords for all users. Normally, this wouldn't be a problem - I'd go to my repository of code, divided at present by year in a "Code05," "Code06," "Code07" format and update a constant in my Constants class and *presto* the connectivity would be back.

Problem is, what if I have to go back more than 3 years?

I'm usually not that irresponsible. In fact, I'm fiendish about backing things up. Yes, I use source control, even on my own little-bitty projects that matter to no one but me. But my mistake was using an external drive - a flakey one - which died after I'd had to reformat/reinstall. I'd backed everything up to it, and my data was gone.

Edge case since I'd after the reinstall I'd moved what was important back to the machine. Edge case because it was in a "code archive" that was something like 6 years old - code that was so long forgotten it was like a head bludgeon when I got a phone call from the client saying that their site was down.

After panicking and looking for some accidental backup (why do we look for things when we know they're gone for good?) I had an epiphany.

I downloaded the *.aspx pages and the site dll - I then used Reflector to disassemble and - I kid you not - 15 minutes later the client was up and running.

And yes, I kept the source...

}

Monday, June 25, 2007

Mac History

{

It's strange that I've had a long relationship to Apple's products - or perhaps not if you're a believer and consider their reach unremarkable...

Growing up in Nairobi, a missionary's kid next door had an Apple IIc he'd occasion to let me use (more often I'd simply watch him using it). A few years later, I spent time in my high school computer classes using Apple IIe computers - for the most part it was typing but I had moments of the extracurricular.

My freshman year of college (1993), after many months hovering around the computer store, drooling and daydreaming, a family gifted my sister and I with a Powerbook 160. The little Powerbook that could took us through the college years although by the end it was on its last legs -

... after which I took a long departure from the Apple universe. I began working with PCs and forgot how much fun it was to pick icons, leverage a trackball, and use the Finder.

After talking my boss into it, however, a year ago I got the priviledge of using Macs again with a company owned, David leveraged iBook G4. It's still a foreign environment but it's nice to return to - especially last week after my HP laptop needed to be reinstalled after catching a virus.

All this to post that there is this interesting graphic of Macs in time, with most of my old friends along the way.

}

Yegge, NBL

{

John Lam reports that Steve Yegge revealed a Javascript implementation of the Rails framework at this weekend's "Foocamp." Having just finished Yegge's most recent post Rich Programmer Food this weekend, and following his NBL (Next Big Language) post from a while back, the dots seem to be connected. It seems early even among technical blogs but I suspect information will start to seep out in the short future.

}

Thursday, June 21, 2007

NDepend vs Me

{

I didn't manage a blog post during TechEd but I'll take a stab at a few things that have swilled in my mind since then. The first thing I've been working on is understanding NDepend, the tool for static analysis. I went to a "Birds of a Feather" spearheaded by some folks from Corillian and had to keep my mouth shut tight so as not to look a fool. Luckily many of the folks there were like me: they knew about static analysis as a concept but were trying to figure out how they could make good use of it.

It will be quite some time before we have static analysis as a part of our build process but I'm most interested in using a tool like NDepend to take the emotion out of code reviews. That is to say that while I do love a vigorous discussion on style and preference, there are concrete measures one can look at objectively to evaluate the well being of software design.

While I'm shaky on the exact meaning of all the metrics (it was suggested to run repeatedly looking for trends) I ran it against a project that has pretty much taken my thirty first year on this earth. For a while that's been on my "to do" list but I think there's always a bit of hesitancy on my part when I'm about to be brutally honest with myself; I designed this software and wrote quite a bit of the code.

The results weren't great, but they weren't horrible. When I used it on the libraries by themselves, the visualizations of the dependencies seemed clean and tidy, and many of the metrics weren't too badly outside of some of the pointers in the cheat sheet we got.

However.

There are some obvious weaknesses that came to light. First and foremost, we were solidly in the zone of instability for most of our assemblies. There are two things that were suspicions now confirmed: first, we didn't have a very formal design process. I need to get better at perceiving my job at an architect level versus as a coder. The second is that we rushed. The rush was not just a schedule thing, it can also be attributed to our short release cycles. The agile folks recommend these, but it should be balanced with a period of silence at the beginning when overall design decisions are being made. The final item, which NDepend would have helped us with in a continuous integration cycle, was showing unused code. After a year's worth of work it's hard to look at so much and except oneself to clean it up, but as a weekly task it would be an easy way to keep things healthy and tight.

Final note: running NDepend is ridiculously easy. The hardest thing besides looking at metrics and trying to understand them is having the courage to look objectively at what you've done.

}

Slowly Back

{

After a weekend with a virus, and a horrible waste of time, I've got a clean installation of XP. The upside was that it forced me back to the iBook G4 that's been a little lonely without me.

}

Wednesday, June 06, 2007

Programming Personality

{

This programming personality test was interesting.

Your programmer personality type is: PLSC

You're a Planner.You may be slow, but you'll usually find the best
solution. If something's worth doing, it's worth doing right.

You like coding at a Low level.You're from the old school of programming
and believe that you should have an intimate relationship with the computer.

You don't mind juggling registers around and spending hours getting a 5%
performance increase in an algorithm.

You work best in a Solo situation.The best way to program is by yourself.
There's no communication problems, you know every part of the code allowing you
to write the best programs possible.

You are a Conservative programmer.The less code you write, the less chance
there is of it containing a bug. You write short and to the point code that gets
the job done efficiently.


}

Tuesday, June 05, 2007

Free Powershell Book

{

All I can say is that I'm loving powershell right now. Hopefully some more goodies will ensue upon this blog but if you're learning like me you can get a free book by leveraging the full length help that is offered on objects. You can print in the following steps:

Get the cmdlets and send the documentation of each to its own file:
get-command % {man $_ -full >"C:/Power/$_.txt"}

Now make an index page so you can navigate to the individuals:
$cmd = get-command % {write-output "<a href='$_.txt'>$_</a><br>"}
"<html>$cmd</html>" >C:/Power/index.html


}