Classpath struggle, even the Jedi

{

Larry O'Brien, well known Jedi Master of many things development posts:

"I have to wire up a ColdFusion to an Axis Web Service. I've spent the past 3
hours trying to figure out freaking classpath issues: something about a
ClassCastException from a org.apache.commons.logging.LogFactory. I'm giving up
for the day. Stupid freaking classpaths."

I can remember sitting in my one room apartment above a garage in Whittier fighting with classpaths, trying to wrangle them into my understanding of Java. It is not schadenfreude that makes me post this, it's more the comfort of knowing that I'm not the only one to have struggled with them.

}

Friction

{

Reading Joel Spolsky and others on the hring process is always interesting. One aspect of their writing that has always intrigued me has been the "quiz questions" used to get a feel for the interviewee's level of intelligence.

Lately I've been interviewing people regularly and am slowly coming up with a format that gets me closest to a person's technical level as it pertains to our company. I usually ask a person to rate themselves on a scale of one to ten in three areas:
1. Web Application Development (HTML / CSS / Javascript / ASP.NET (or any other web app framework or perl/scripting env)
2. "Fat" clients (Windows Forms .NET / COM )
3. Database programming and administration (SQL / Database Design)

I then follow up their ratings of themselves with some general questions in each area. For example, most people consider themselves knowledgable with web development because they know some basic html. Questions I like to pose here may be along the lines of asking what the difference is between DIV and SPAN elements, or what their understanding of why XHTML (versus HTML 4.0) came into existence, or whether they think designing with tables is a bad idea. On the programming side I like to ask a person about the postback cycle in ASP.NET, or if they come from a non-.NET environment, about something like state management (querystring, session, etc...) in web applications. Another favorite area of mine is javascript since many people think they know it but can't provide answers for even a simple question like how to register an event for an html element. When I ask questions the point is definitely not about trivia but more to gauge whether the candidate is about aptitude or whether they'd be capable of jumping right in the fray and making a contribution.

But beyond the technical questions, beyond the idea of "Smart, Gets Stuff Done" as goals in the ideal candidate, probably the most important thing about working the environments I've been exposed to is learning how they deal with "Friction." I was watching a panel show once on CSPAN where a wizened old military man said: "The difference between war plans and war is friction." As I understand it, friction has to do with all those little things (and big things) that go wrong. In our case, friction is almost always involved in trying to hit release dates. Friction is also a big player when it comes to team dynamics and how people treat each other.

For all the talk online about people who are exceptionally smart and productive, I'm surprised that the question of friction isn't more important when choosing programmers since friction is always a part of software development. Everyone's got a story about this, but here's mine: I worked with a guy who was exceptionally sharp, anal-retentive, and detail oriented. He'd forgotten more within his discipline than I probably will know. I would ask him from time to time for help on the little problems I was dealing with and his answers were always prompt and accurate. We worked together as peers and that's what made me capable of liking him however, for many people, he was nearly impossible to deal with because any change that required flexibility on his part was out of the question. If I'd been a team lead on a project, he'd be a difficult decision because of this attitude, and what would push me from the "maybe" into the "no" decision would not be my own ability to work with him, it would be his affect on the team as a whole and how he affected the team's group culture.

One thing to bear in mind is that I'm a consultant who writes business-ware which may mean that friction comes up for me a lot more than someone working at a shrinkwrap company, or a behemoth powerhouse with zillions of great hackers. But as my experience accumulates, I begin to see that a person who has the ability to calm a client down while being able to refocus and get a project out the door is more valuable than a person who is a brilliant programmer but unable to deal with the type of rapid change and improvisation necessary for our work.

}

Commands and Abstractions

{

I clash with people occasionally over my approach to SQL Server. My approach to SQL Server is quite simple: Transact SQL. Never one for layers of abstraction and always seduced by speed, I prefer to keep my management activity in Query Analyzer. SQL Management Studio seems a bit slower than its older cousin, so quite often I prefer Query Analyzer even with the brand new SQL Server 2005.

Often times for me preferences like this develop after bad experiences but today was a reminder of what kinds of things a layer of abstraction is capable of doing behind your back if you don't monitor it. We have a table with 82 columns and something like 50,000 rows. One of these columns needed to be extended from VARCHAR(10) to VARCHAR(50). I was walking through the steps as they may occur in Enterprise Manager so I opened the product (wait a few seconds), selected my database (wait a few seconds), opened the tables node (wait a few seconds), and then right clicked on my table to examine and manipulate the column (another few seconds, plus scrolling up and down to locate the column I wanted). I made my change and then proceeded to make a "change script" which, to my horror, creates a temp table, copies data and affects other tables referenced by the table whose column I was changing.

My usual approach to something like this is the following: Start->Run (Instantaneous). "isqlw" and hit enter (Instantaneous). In the query window I type the following:

1 USE MyTestDatabase

2 GO

3

4 ALTER TABLE Foo

5 ALTER COLUMN Bar VARCHAR(50)

6 GO

This is a simple example, titled away from the GUI but there's more to this. Because the GUI represents an abstraction, there are limitations a person who is there encounters and begins to operate with. A simple example is the following: assume a table has 3 stored procedures (select, insert, update) and you've added a few columns that you'd like to include as parameters in each of the procs. In Enterprise Manager, or SQL Management Studio, a person is inclined to think in terms of a single procedure at once and likely to visit each procedure to make this change. A wise Unix gray beard or someone else who's comfortable with a text editor will dump all the procedures they need to modify into a single file, do some search/replace operations as a single step, and do the update.

The point for me really veers from SQL Server at this point and gets into the question of the abstractions which I myself rely on unnecessarily - which prevent my thinking to get past a certain point. Although I have come to rely on the command line more and more over time, there are still things I learn: DOS things! that have really changed how I think about certain types of problems. A simple example of this is from a few months ago when I learned of the existence of pushd and popd, commands for storing and retrieving working directories on a stack. These relatively trivial techniques make it easy to do things that never really occured to me before. Like the fact that I could just use fc to check for differences in files. Or piping a request for help to a file so I could have documentation for a command. Here is an example that might be interesting: wsdl.exe /? >wsdlParameters.txt

One last point of clarification: I am not against abstractions. In many instances they work quite well. A while back I wrote a web based tool for viewing objects in SQL Server databases - basically outputting queries to the Information Schema views. Everything that tool does can be duplicated in query analyzer or in SQL Management Studio. But in terms of raw speed, flipping back and forth on table definitions in the web based tool beats clicking around in the GUI or retyping the same select statement with a different table name in the where clause. What I believe is that abstractions are used best when what they are hiding is understood fully. This is counter intuitive; most people think the abstractions - an easy GUI, a simple command, and so on - are there to make things simpler for people who want to keep details hidden. That is always dangerous, even when it's effective. I'd prefer to struggle with what's underneath and then live blissfully choosing the abstractions I find useful.

}

Serious Reading

{

Chris Sells answers a few questions in public but one comment got my attention:

How are you able to keep up with the changes? What books do you read?

[csells] I keep up with changes by a) a broad familiarity with as much technology as possible and then b) committing to using it because it feels like it’ll be the right thing and c) using fear to motivate me (recognize a pattern? : ). I read books on demand given the topic I’m into, and then it’s 3-5 books in a week for immersion. Frankly, after writing a few books, I’m a bit of a snob, so I don’t read a lot of technical books for fun the way I used to.

Wow: three to five books in a week for "immersion." It's a stark reminder on a personal level of how much more aggressive I need to get with my own reading list (which, incidentally, includes Sells's Windows Forms Programming) but in a general sense it shows a hallmark of what the information age does to programmers - it might not be intelligence that separates us (though I think Sells is remarkably intelligent), it's the ability to absorb information quickly.

I don't read very quickly so it's a matter of discipline and will to work on absorbing what's relevant for the job and the future. But in a word where guys like Sells chew up and spit out technologies before most people are seriously working with them, a person needs a good reminder of staying focused.

It reminds me of the brief exchange I had with Hanselman. I'd ask a question, he'd casually suggest a book. It says, to me, that there's a bit of mysticism to how smart people like that stay on top of things but really that the information is out there and ready to be consumed if only one is disciplined enough to pursue it.

}

Windows Forms for Visual Basic 6 users

{

My current project at work is a Windows Forms application. Even though I've been programming .NET almost daily since it's introduction, just about all of what I've done is web based programming using ASP.NET. The non-web code I've written has been for the most part libraries (DLLs) to use in web apps.

Most of what I know about thick clients comes from doing work in Visual Basic 6 so while I usually know where I'd like to go with something, I find myself hitting a wall with how to do it in VS 2005.

An example of this would be this morning, trying to figure out whether the Locked property for controls is ported to .NET Windows Forms. Unfortunately it's not - the Locked property you find has to do with resizing and moving, not making controls editable.

Fortunately MSDN has a great resource here for people making the transition that lets you see Visual Basic 6.0 and Windows Forms compared directly.

}

Rhino Book

{

I got David Flanagan's new Javascript Reference earlier this week. My fourth edition is nice and worn, but it will be nice to have the updated perspective since so much has happened in internet time since the fourth edition was published. It's the third edition of the book that I've owned and while buying the same book more than once has a big psychological drawback, I'm especially looking forward to reading the sections on the XmlHttp (AJAX) side of javascript.

}

Firefox Hiding Namespaces! ARG.

{

For a while I've had people ask me how I did my photoblog and because I wrote it from scratch it's not really useful (and it's definitely "Me-ware": unusable to just about anyone but me). I always direct people towards Flickr but it doesn't have the photoblog experience quite like mine and other photoblogs.

This weekend I've been writing a tool called Flickr-Fu that grabs the RSS feed from Flickr and attempts to simulate the photoblog experience allowing a user to navigate forward and backward one photograph at a time. The RSS xml structure is extended by Yahoo with it's own namespace: http://search.yahoo.com/mrss - all such tags are given a "media" prefix to distinguish them from the structure of RSS 2.0. Using XPaths with .NET is pretty straight forward until you run into the namespace issue, at which point there is a class XmlNamespaceManager that you will need to tame in order to get the job done.

But before this point I was hitting my head against a wall: I could see the media:content tag that I needed to get the image URL but I could see no namespace associated with it! It doesn't follow as well-formed XML to have namespaces without some namespace URI but sure enough, Firefox was hiding the namespace URI. I pasted to IE and was able to get a proper reference to use with the XmlNamespaceManager. What a strange thing to display XML in a stylesheet but leave off the namespace references in the root element! Just to prove I'm not hallucinating, here's a screen cap:

Once the namespace issue was cleared up for me, the code was quite simple:

21 XmlNamespaceManager spaceManager = new XmlNamespaceManager(doc.NameTable);

22 spaceManager.AddNamespace("media", "http://search.yahoo.com/mrss");

23 linkToBig = xnItem.SelectSingleNode("media:content/@url", spaceManager).InnerText;

Flickr-Fu is a work in progress but you can have a look here.

}

Raganwald Responds

{

I had posted about Yegge's Agile/agile essay, trying to follow some of the interesting comments that ensued on the web. I made the following comment about Raganwald's response:

"Raganwald has the most interesting response
I've seen so far: that it all comes down to people over process and a good team
will succeed despite methodological strategy. "

He has since posted an elaboration in response on his blog - I was a little off in my reading:

"No. I'm saying that a "good" methodology cannot save a bad team. That does not
mean that a good team can survive a bad methodology. There are four
critical parts of a successful software project, and in my experience you
cannot succeed if any of them are missing. People are the first of those,
management the last. Methodology is a part of management in my view. In some
situations, strong management can impose an informal rather than a formal
methodology. But it would be stretching a point to claim that such situations
lacked process or had a bad process."

}

NULLs in subqueries

{

Here was an interesting issue we ran into this week. I haven't run into this before and thought I should share - a friend was using a subquery to filter and because of NULL results in the field his result was an empty set.

My approach to things like this is usually to use a join rather than a subquery and ultimately that's how we saw how Microsoft SQL Server was processing things. But this behavior is quite subtle so I'm posting since a person running into the same issue may Google there way to this entry:

1 CREATE TABLE #TRAN(

2 TRANID INT,

3 TRANNAME VARCHAR(50)

4 )

5 GO

6

7 CREATE TABLE #ACH(

8 ACHID INT IDENTITY(1,1),

9 ACHNAME VARCHAR(50),

10 TRANID INT NULL

11 )

12 GO

13

14 INSERT INTO #TRAN VALUES(1,'TRANSACTION A')

15 INSERT INTO #TRAN VALUES(2,'TRANSACTION B')

16 INSERT INTO #TRAN VALUES(3,'TRANSACTION C')

17 INSERT INTO #TRAN VALUES(4,'TRANSACTION D')

18 INSERT INTO #TRAN VALUES(5,'TRANSACTION E')

19

20

21 INSERT INTO #ACH VALUES('ACH 1A', 1)

22 INSERT INTO #ACH VALUES('ACH 2A', 2)

23 INSERT INTO #ACH VALUES('ACH 3A', 3)

24 INSERT INTO #ACH VALUES('ACH 1B', 1)

25 INSERT INTO #ACH VALUES('ACH 1C', 1)

26 INSERT INTO #ACH VALUES('ACH 3B', 3)

27 INSERT INTO #ACH VALUES('TEST',NULL)

28

29 -- RETURNS NOTHING

30 SELECT * FROM #TRAN

31 WHERE

32 TRANID NOT IN

33 (SELECT TRANID FROM #ACH)

34

35 -- RETURNS TRANID 4,5

36 SELECT * FROM #TRAN

37 WHERE

38 TRANID NOT IN

39 (SELECT TRANID FROM #ACH WHERE TRANID IS NOT NULL)

}