Twy

{

I'm still trying different things with Twining.  I'd thought about writing some "front end" type experience for usage but it really crystallized as a need when I showed it to a person I know and there seemed to be a disconnect in how it could be used.  For me it's natural to set my path environment variable, launch favorite text editor X and then run things from the command line or a script, but it's a nuisance if you're used to a one stop shop for being able to use some tool.  And as much as I want language as the focal point in the word "tool" there is something practical in the notion of something you download and click a button to execute with.

Enter Twy, which I pieced together after looking at a few samples of a hosted DLR engine in a Windows Forms app.  Now one need not figure out how to install or configure anything, or worry about creating and disposing of script files.

If you want to write something that hosts the DLR engine, take a look first at these samples on Voidspace.  There are other samples online if you hunt and peck but be aware that things have changed between the various releases of IronPython.  A few gotchas for me:

1. Redirecting standard output:

// where engine references the ScriptEngine type
// and ms references a Stream of some sort
engine.Runtime.IO.SetErrorOutput(ms, Encoding.UTF8);
engine.Runtime.IO.SetOutput(ms, Encoding.UTF8);

Many examples of this are deprecated for the IronPython 2.x beta

2. Referencing classes in mscorlib:

Be aware that doing the following:

import clr
clr.AddReference("System")

is not going to be enough to get types out of mscorlib.  Although types will load from System, you'll need to get a reference to the assembly directly if you plan to use it in your hosted engine.  I had a little trouble with the StringBuilder but easily resolved it with the following after a tip on the IronPython mailing list.

Assembly assem = Assembly.GetAssembly(Type.GetType("System.Text.StringBuilder"));
scope = engine.CreateScope();
engine.Runtime.LoadAssembly(assem);

3. The only novel thing I did that I didn't see a lot of was loading a module so that you could utilize it with your hosted engine.  I added Twining.py to the project and set Visual Studio to copy it to the compile destination.  I then have the following code which keeps the module available for later use:

string p = Path.Combine(Environment.CurrentDirectory, "Twining.py");
scope = engine.Runtime.ExecuteFile(p); 

// later on:

ScriptSource source =
    engine.CreateScriptSourceFromString(input,
        SourceCodeKind.Statements);
object res = source.Execute(scope);

All in all not rocket science, it's amazing how much power one has at their fingertips in such a small application.  I would love to see other modules, especially ones that define some interesting type of DSL, have utilities like this that let you play around without much effort.

Oh yeah, the project and source.  Download it here, I'll clean up a bit more later.

}

Getting better with meta-thinking

{

Some excerpts from a great post from Ola Bini, one of the JRuby core developers:

In short, I believe that being able to abstract and understand what goes on in a programming language is one way to become more proficient in that language, but not only that - by changing your thinking to see this part of your environment you generally end up programming differently in all languages...

A little further on, emphasis is mine:

... I call this meta-level thinking. I think it's mostly a learned ability, but also that there is an aptitude component.

Cheers for this as a "learned ability" which would give one pretentious enough to call his blog "Metadeveloper" some hope.

}

Generic Enum Parsing in C#

{

Generics, as it were, are passe to talk about these days. However, I found myself dealing with enumerations on Friday and was a little surprised there wasn't an approach to parsing these out in a more generic fashion. I scribbled the following, perhaps it will be of value to someone:

enum Test { 
    v1, 
    v2,
    v3   
}

public static T EnumParser<T>(string givenValue)
{
    return (T)Enum.Parse(typeof(T), givenValue, true);
}

//usage:
Test val = EnumParser<Test>("v1");

}

Sense of urgency

{

One thing I’ve come to realize is that urgency is overrated. In fact, I’ve come to believe urgency is poisonous. Urgency may get things done a few days sooner, but what does it cost in morale? Few things burn morale like urgency. Urgency is acidic.
- read the whole post

Jason Fried of 37Signals neglects to mention one aspect that seems to come back over and over to haunt those of us who live with "urgency"; that it costs dearly to rework things that were done in a hurry.

}

From TFS to SVN

{

My project at work added some remote developers and it was decided that rather than try to figure out TFS, which we use for everything internally, we'd use SVN. I'm not used to SVN since we used Team Foundation Server for source control and Visual SourceSafe before that - I've used TortoiseSVN quite a bit to get source code from projects online, but never as a primary source control environment for a big project.

So far it's a breeze. I installed VisualSVN Server and configured repositories and users - this took about 10 minutes.  After that I was up and running with TortoiseSVN in about 3 or 4 minutes.

Hm...

Because in a parallel universe a completely different project I'm working on required me to install the TFS client for Visual Studio 2008.  On my super duper 4GB Ram, 3 Ghz, desktop machine at work it took about 45 minutes.

Just the client.

So yeah, I've been thinking about that contrast quite a bit all day.  One thing I think is interesting is the contrast between small teams as I heard discussed at CodeMash earlier this year and the features of TFS: all the note taking, task assignment, iteration,  etc, etc... On a "one to two pizza" team (e.g. a Google, Amazon), how many people go in depth with those features versus some massive team of developers (e.g. a government agency)?

}

Twining

{

Now has a its own location. I'll still post of goings on, but will keep the external site as a source of updates and documentation. There's a form there for feedback as well.

}

From Komodo to Wings

{

I started my IronPython musings with Notepad++ since it's my usual text editor (after being enamored with Textpad for many years, I moved on - lots of reasons, perhaps I'll blog them later).  As time passed and my scripts grew in size I started using an old version of Komodo I've got which has decent support for Python. I used Komodo (3.5) when I was writing a lot of Perl and it worked pretty well for that, especially since it had a good enough debugger and CGI emulator for me to dig myself out of holes.  Komodo does okay with Python but I've been using Wings IDE on recommendation from fuzzyman and it's much more responsive and fluid as a development environment.  Additionally, I got Komodo when there was still a cheapo version but now it's quite expensive - exceeding the Wife Acceptance Factor for experiments (WAFe).  Wings is quite reasonable, I'll start with the Personal edition and if I can get my company onboard I'll ask that they upgrade my license.

Unfortunately I still need to run my scripts outside the IDE environs since Wings isn't built with IronPython support (correct me if I'm wrong), but I'm quite used to this now. 

}

Programming Languages, Trends

{

DDJ (the print edition at least) has an interesting interview with Paul Jansen, managing director of TIOBE on languages. Some interesting points:

• Direct quote: "I expect in five years time there will be two main languages: Java and C#, followed closely by good old Visual Basic."
• While not much has changed over the last 10 years, COBOL is now out of the top 10 list and Python is moving in.
• Direct quote: "Another langauge that has had its day is Perl. It was once the standard language for every system administrator and build manager, but now everyone has been waiting on a new major release for more than seven years. That is considered too long."

Interestingly enough, TIOBE has picked the language of the year to be Python.

}

Reddit for Javascript

{

Via Raganwald, there's now a Reddit for JavaScript.

}

Generate Catalog Images in IronPython

{

What if you had a client who has a big product catalog you're integrating with the web.  You work out a convention for how data is displayed and want to generate some test images for yourself...

First, just some basics on generating a single image:

import clr
clr.AddReference("System.Drawing")
from System import *
from System.Drawing import *

def GenerateImage(text):
    starter = Bitmap(1,1)
    g = Graphics.FromImage(starter)
    f = Font("Arial", 14) 
    theImage = Bitmap(starter, Size(200,200))
    g = Graphics.FromImage(theImage)
    g.Clear(Color.Orange)
    g.DrawString(text, f, SolidBrush(Color.DarkBlue), 0, 0)
    g.Flush()
    theImage.Save("c:\\temp\\pyImage.gif")    

if __name__ == "__main__":
    GenerateImage("Holly was a hunter")

Here's some TSQL for the sample data I'll use:

create table testdata(
    testid int identity(1,1), 
    testvalue varchar(50)
)

declare @i int
set @i = 0
while @i < 25 begin
    insert into testdata(testvalue)
        values('PRODUCT ' + convert(varchar(2), @i))
    set @i = @i + 1
end

Now for a finalized version that hits the database and generates images:

import clr
clr.AddReference("System.Drawing")
clr.AddReference("System.Data")

from System import *
from System.Drawing import *
from System.Data import *
from System.Data.SqlClient import *

def GenerateImage(text, path):
    starter = Bitmap(1,1)
    g = Graphics.FromImage(starter)
    f = Font("Arial", 14) 
    theImage = Bitmap(starter, Size(200,200))
    g = Graphics.FromImage(theImage)
    g.Clear(Color.Orange)
    g.DrawString(text, f, SolidBrush(Color.DarkBlue), 0, 0)
    g.Flush()
    theImage.Save(path)
    
def GenerateImagesFromDatabase(connectString, query, sourceColumn, outPath):
    cn = SqlConnection(connectString)
    cn.Open()
    cmd = SqlCommand(query, cn)
    r = cmd.ExecuteReader(CommandBehavior.CloseConnection)
    while r.Read():
        baseText = r[sourceColumn].ToString()
        GenerateImage(baseText, outPath + "\\" + baseText + ".gif")
    r.Close()

if __name__ == "__main__":
    connect = "Data Source=.\\sqlexpress;Initial Catalog=MVCBaby;Integrated Security=SSPI;"
    sql = "select * from testdata"
    col = "testvalue"
    GenerateImagesFromDatabase(connect, sql, col, "c:\\temp\\fobot")

 

Pretty cool stuff, I hope it's useful to more people than just myself.  Download the first script here, the second more elaborate one here.

}

Twining

{

I've been quiet but still working on Twining.  Unfortunately lots of distractions at work have slowed me down. Most of what I've done of late is adding unit tests for the functionality that already exists.  I twittered that adding unit tests a posteriori is not fun and tends to have the effect of one simply verifying what they've done with pasted code.   Now that most of that is done I'd like to add a few things:

1. Operating on queries like tables for export and copy.  Everywhere that you see "table" below, I'd like to have "query":

database(cn).table("MyTable").transform(mytrans).copyto.database(cn2)
database(cn).table("MyTable").transform(mytrans).copyto.xmlFieldsAsAttributes("c:\\here.xml")
database(cn).table("MyTable").transform(mytrans).copyto.xmlFieldsAsElements("c:\\there.xml")
database(cn).table("MyTable").transform(mytrans).copyto.delimited("\t", "c:\\foaf.tsv")
database(cn).table("MyTable").copyto.delimited("\t", "c:\\foaf_no_trans.tsv")
database(cn).table("MyTable").transform(mytrans).copyto.csv("c:\\foo.csv")
database(cn).table("MyTable").copyto.xmlFieldsAsAttributes("c:\\here.xml")
database(cn).table("MyTable").copyto.xmlFieldsAsElements("c:\\there.xml")

how about: 

database(cn).query("select this, that, [the other] from foo").copyto.csv("c:\\querydata.csv")

2. Support for SqlBulkCopy - as far as I know this is only available between Sql Servers (or, more specifically SqlConnection instances).

3. Cloning of tables - if you use sql server, this is quite simple with the following TSQL:

SELECT * INTO MYCOPYTABLE FROM MYORIGINALTABLE WHERE 1 = 2

4. Database "ping" - just a simple way to test connection strings.

cn = "my connection string"
database(cn).ping() 

5. Inspired by ConfigObj I'm wondering what a JSON like table definition would look like. In ConfigObj, you can define things like:

section2 = {
    'keyword5': 'value5',
    'keyword6': 'value6',
    'sub-section': {
        'keyword7': 'value7'
        }
}

I wonder if one could have the following for a table:

tableDef = {
    'personid': {
        'datatype':'int',
        'identity':'true',
        'seed':1        
    },
    'firstname':['varchar',50, 'null']
}

Just something to gnaw on...

 

I'll hopefully get a chance to work on this stuff soon.

}