A proposal to simplify calling L# from C#

Note: this has also been posted to its own thread in the LSharp Google Group. If you want to discuss the merits of this proposal, please consolidate all comments there. Thanks.

Calling L# from C#

Right now, it's much harder than it needs to be to call L# code directly from C#. Since one obvious use of L# is as a scripting language for .Net applications, it would be nice if it was really simple. Of course, one good way to make this happen would be with an L# compiler that produces a .Net assembly that can be called from any .Net language. However, as useful as this would be, it still might not be ideal for some application scripting scenarios. In some situations, it may be more appropriate to use the interpreter (which already exists), and simplify the interface to make it easier to call from C# and other languages.

Suppose you have a lisp environment already running, and you want to call a simple function you have defined. The easiest way to call it right now is probably to create an L# string and evaluate it, like so:

object result = Runtime.EvalString("(my-function)",
    new LSharp.Environment());

This isn't a very clean way to work with code, but it should work in simple cases. In more elaborate cases, building correct strings gets to be an increasingly elaborate problem. And really, even if it weren't that hard, marshalling data to and from strings for interoperation isn't an especially good practice.

The other way to call L# code from C# is to work with the actual L# objects: symbols, functions, closures, and so on. Here is some simple example code for calling the system function map and the user-defined function (technically, a Closure object) concat from C#:

LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"),
    env));

// build the arguments for map
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env.GetValue(Symbol.FromName("double")));
Cons intCons = Cons.FromArray(new object[]
    { 1, 2, 3, 4, 5 });
myarraylist.Add(intCons);

// call map
Cons args = Cons.FromICollection(myarraylist).Reverse();
Function map = (Function)env.GetValue(
    Symbol.FromName("map"));
Console.WriteLine(map(args, env));

// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
args = Cons.FromArray(myarray);
Closure concat = (Closure)env.GetValue(
    Symbol.FromName("concat"));
Console.WriteLine(concat.Invoke(args));

The library that is being loaded, mylib.ls is simply:

; mylib.ls -- simple sample library
;
; Copyright (c) 2006, Andrew Norris
; Simple license: reuse this however you like.

(= double (fn (x) (* x 2)))

(= concat (fn (&rest items)
    (let sb (new system.text.stringbuilder)
      (foreach elem items
        (call append sb elem))
      (tostring sb ))))

When the above C# code executes, it will output:

(2 4 6 8 10)
foobar42baz

Using the L# interpreter objects directly is a better idiom in general, because it doesn't require data structures to be marshalled as strings, and because it allows for some things to be checked at compile-time. However, it's obvious that this code is pretty cumbersome, if all you want to do is call a couple of functions.

A side note: the ability to call a couple of trivial functions like this generally isn't worth the trouble of embedding the L# interpreter in your C# application. However, there are some things that may be much easier to write in L#, such as list manipulation functions or code that uses macros effectively.

1. Add a string indexer to Environment

It would be nice if it were easier to access symbol values in the L# Environment. C# doesn't have symbols, so you have to create one from a string. The GetValue call adds some an additional operation as well. This seems like a good place to start in making it simpler to call L#.

The current method of dereferencing a symbol isn't that complex an operation if it's called occasionally, but working with symbols in the environment is one of the two most basic operations in L#, along with calling a function. To be usable, it needs to be concise. A string indexer makes this code much simpler:

LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));

// build the arguments for map
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env["double"]);
Cons intCons = Cons.FromArray(new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intCons);

// call map
Cons args = Cons.FromICollection(myarraylist).Reverse();
Function map = (Function)env["map"];
Console.WriteLine(map(args, env));

// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
args = Cons.FromArray(myarray);
Closure concat = (Closure)env["concat"];
Console.WriteLine(concat.Invoke(args));

As you can see, it doesn't affect the rest of the code, but it makes recovering the functions from the environment much simpler.

Adding a basic string indexer to Environment is simple. The code looks like this:

public object this[string s]
{
    get { return GetValue(Symbol.FromName(s)); }
    set { Assign(Symbol.FromName(s), value); }
}

Of course, this only gets things started. There are a lot of other things we can improve as well.

2. Simplify the interface for converting a .Net data structure to a Cons

The interface for converting a collection of items to a Cons can usefully be simplified. Cons contains some useful conversion functions, and SpecialForms.The() does a good job of wiring them up. But, really, anything that can be enumerated can be easily converted to a Cons, if we add a new method. Cons.FromIEnumerable() is simple, and virtually identical to Cons.FromICollection():

public static Cons FromIEnumerable(IEnumerable enumerable)
{
    object list = null;
    foreach (object o in enumerable)
    {
        list = new Cons(o, list);
    }
    return (Cons)list.Reverse();
}

This doesn't have a significant impact on the code right away. But it will enable us to do some more important steps later on.

LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));

// build the arguments for map
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env["double"]);
Cons intCons = Cons.FromIEnumerable(new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intCons);

// call map
Function map = (Function)env["map"];
Cons args = Cons.FromIEnumerable(myarraylist);
Console.WriteLine(map(args, env));

// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
args = Cons.FromIEnumerable(myarray);
Closure concat = (Closure)env["concat"];
Console.WriteLine(concat.Invoke(args));

It also enables you to convert any class that implements IEnumerable:

Cons cons = Cons.FromIEnumerable(someRandomThingWithAnEnumeration);

The advantage of this is that whenever something new comes along that needs to be turned into a Cons, it can already be done if there is any standard way to reference all the items.

3. Make it easy to convert nested .Net data structures to Conses

After creating the code to consistently convert enumerable data structures to Conses, one obvious problem is that it won't automatically handle nested data structures. In the following code, the array and the ArrayList have to be converted separately:

ArrayList myarraylist = new ArrayList();
myarraylist.Add(env["double"]);
Cons intCons = Cons.FromArray(new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intCons);

Cons args = Cons.FromICollection(myarraylist).Reverse();

Since nested lists -- often, deeply nested ones -- are one of the most basic building blocks of Lisp code, it would be nice if we could convert the whole data structure at once. Forutunately, if we extend the FromIEnumerable method we just built, there's a straightforward solution:

public static Cons FromIEnumerable(IEnumerable enumerable, bool isRecursive)
{
    object list = null;
    foreach (object o in enumerable)
    {
        if (isRecursive && o is IEnumerable) {
            o = FromIEnumerable(o as IEnumerable, true);
        }
        list = new Cons(o, list);
    }
    return (Cons)list;
}

Now we can easily convert the data structure to a nice, Lispy nested list in one step:

ArrayList myarraylist = new ArrayList();
myarraylist.Add(env["double"]);
int[] intArray = new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intArray);

Cons args = Cons.FromIEnumerable(myarraylist, true);

This doesn't have a significant impact on our sample code here, but for cases where there is data that is already in elaborate .Net data structures and needs to be passed to L#, it will make things significantly easier, and avoid the need to walk the data structure tree.

4. Simplify building Conses in C#

Passing nested data to an L# function from C# is still fairly cumbersome. While building arrays and ArrayLists is often more natural for working with C# data, it's cumbersome for packaging up arguments, as we saw in the last section when we built the arguments to pass to map. Consider the code we've just been looking at:

ArrayList myarraylist = new ArrayList();
myarraylist.Add(env["double"]);
int[] intArray = new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intArray);

Cons args = Cons.FromIEnumerable(myarraylist, true);

This could be done differently by building a Cons directly, of course:

Cons intCons = Cons.FromArray(new object[] { 1, 2, 3, 4, 5 });
Cons args = new Cons(intCons);
Cons args = new Cons(env["double"], args);

This is simpler, but still harder than it needs to be. Also, it either involves performing the steps out of the usual order -- this is normal in Lisp, but unusual in C# -- or adding a Reverse() operation each time.

By contrast, in L#, the equivalent code is simply:

(map double '(1 2 3 4 5))

Obviously, if we could simplify the C# version, it would be a lot easier to package arguments and call L# functions. For example, this code would be much closer to ideal:

Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));

That can be implemented relatively straightforwardly by creating a Build method that can take any number of arguments and simply build a list out of them:

public static Cons Build(params object[] items)
{
    Object cons = null;
    for (int i = items.Length - 1; i >= 0; i--)
    {
        cons = new Cons(items[i], cons);
    }
    return (Cons)cons;
}

This simplifies the example code to:

LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));

// call map
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
Function map = (Function)env["map"];
Console.WriteLine((Cons)map(args, env));

// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
args = Cons.FromIEnumerable(myarray, false);
Closure concat = (Closure)env["concat"];
Console.WriteLine(concat.Invoke(args));

5. Provide an identical interface for functions or closures

Wouldn't it be nice if you could have one simple way to call an L# operation, regardless of whether it was a system function, a user-defined function (closure), or a special form? And wouldn't it be nice if you could pass the operation whatever data structure you had without converting it?

For example, here's what the code to call map and concat might look like if you could do that:

Cons result = (Cons)map(args);

string sresult = (string)concat(myarray);

That's a lot simpler than the earlier example, right? Well, with delegates, it's easy to produce that interface:

public delegate Object fn(IEnumerable arguments);

There's really only one problem with this delegate: it doesn't match the signatures of Closure or Function, so you can't use it, at least directly. Fortunately, there's a way around that problem:

6. Build an implementation class that can make function calls simple

Although the interface to Closure and Function doesn't match the fn delegate signature, it's straightforward to make an adapter class that makes them match. Basically, the idea is to automatically translate data structures to Conses, and automatically bind functions to the current environment. The class then contains a method with the proper signature for the fn delegate, and a property that will return it.

public class FunctionBinding
{
    private Environment _env;
    private Function _f;
    private Closure _c;
    private bool _isClosure;

    public FunctionBinding(Environment env, Function f)
    {
        _f = f;
        _env = env;
        _c = null;
        _isClosure = false;
    }

    public FunctionBinding(Closure c)
    {
        _f = null;
        _env = null;
        _c = c;
        _isClosure = true;
    }

    private Object Impl(IEnumerable arguments)
    {
        // special handling for closures that take no
        // arguments and functions with empty lists
        if (arguments == null)
        {
            if (_isClosure)
                return _c.Invoke();
            else
                return _f(new Cons(null), _env);
        }

        // build a cons out of the argument
        Cons cons = null;
        if (arguments is Cons)
            cons = arguments as Cons;
        else
            cons = Cons.FromIEnumerable(arguments, true);

        // invoke the closure or function
        if (_isClosure)
            return _c.Invoke(cons);
        else
            return _f(cons, _env);
    }

    public fn BoundFunction
    {
        get { return Impl; }
    }

    public static fn Bind(Environment env, Function f)
    {
        FunctionBinding fb = new FunctionBinding(env, f);
        return fb.BoundFunction;
    }
}

This makes it really simple to call L# functions, but complicates retrieving them. The problem is that we have to build a FunctionBinding object around the return value we get from the environment. At this point, our code would work like this:

LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));

// call map
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
fn map = (new FunctionBinding(env, (Function)env["map"])).BoundFunction;
Console.WriteLine(map(args));

// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
fn concat = (new FunctionBinding((Closure)env["concat"])).BoundFunction;
Console.WriteLine(concat(myarray));

7. Wire things up so FunctionBindings are created automagically

Fortunately, we can simplify things. By modifying the string indexer we built earlier, we can automatically bind functions and closures, and return the easy-to-call bound functions.

public object this[string s]
{
    get
    {
        object o = GetValue(Symbol.FromName(s));
        if (o is Function)
            o = (new FunctionBinding(this, (Function)o)).BoundFunction;
        else if (o is Closure)
            o = (new FunctionBinding((Closure)o)).BoundFunction;
        return o;
    }
    set { Assign(Symbol.FromName(s), value); }
}

This will also convert our call to the double function as well, which means the interpreter needs to be able to handle fn delegates:

public static object Apply (object function, object arguments, Environment environment) 
{
    if (function.GetType() == typeof(Function)) 
    {
        return ((Function) function) ((Cons)arguments,environment);
    }

    // If function is an LSharp Closure, then invoke it
    if (function.GetType() == typeof(Closure)) 
    {
        if (arguments == null)
            return ((Closure)function).Invoke();
        else
            return ((Closure)function).Invoke((Cons)arguments);
    }

    if (function is fn)
    {
        return ((fn)function)((IEnumerable)arguments);
    }
    else 
    {
        // It must be a .net method call
        return Call(function.ToString(),(Cons)arguments);
    }
}

After this final change, our code simplifies to this:

LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));

// call map
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
fn map = (fn)env["map"];
Console.WriteLine(map(args));

// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
fn concat = (fn)env["concat"];
Console.WriteLine(concat(myarray));

And if you want, you can simplify it even further, though it makes the code a bit denser:

LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));

// call map
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
Console.WriteLine(((fn)env["map"])(args));

// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
Console.WriteLine(((fn)env["concat"])(myarray));

Conclusion

Implementing these changes should make it much easier to call L# code from inside C#, and without resorting to building evaluation strings. Hopefully, this will make it more practical to take sections of code that can be implemented much more easily in L# than in C# and build a multiple-language implementation. For example, in a program that has a section that requires elabortate list manipulation, but other sections that need to be in C#.

With these changes, we were able to simplify our sample code from

LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));

// build the arguments for map
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env.GetValue(Symbol.FromName("double")));
Cons intCons = Cons.FromArray(new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intCons);

// call map
Function map = (Function)env.GetValue(Symbol.FromName("map"));
Cons args = Cons.FromICollection(myarraylist).Reverse();
Console.WriteLine(map(args, env));

// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
args = Cons.FromArray(myarray);
Closure concat = (Closure)env.GetValue(Symbol.FromName("concat"));
Console.WriteLine(concat.Invoke(args));

to

LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));

// call map
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
Console.WriteLine(((fn)env["map"])(args));

// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
Console.WriteLine(((fn)env["concat"])(myarray));

Of course, to do the equivalent in L#, the code is

(load "mylib.ls")

(prl (map double '(1 2 3 4 5)))

(prl (concat "foo" "bar" 42 "baz"))

So L# functions still aren't nearly as easy to work with from C# as they are natively. Even keeping this in mind, these changes should make it a lot more practical to embed direct calls to L# code in C#.

The main thing this proposal hasn't touched on, of course, is macros. If you have macros embedded in L# functions, this won't make any difference -- the macros get expanded at read time, and your function can be called from C# just like any other function. But what if you want to call a macro directly? Or what if you want to embed a macro call inside the arguments to the function you call?

The techniques discussed in this article won't handle macros correctly, because by the time a function is being called, L# assumes all macros have been expanded. In a future post, I'll discuss how to use similar C# techniques with macros.

Finally, in the future, adding compiled code will be an important way to make L# code more accessible from C#. That will allow L# libraries to be attached at compile time and statically bound, rather than loaded and called at runtime. While that will be a powerful technique for some situations, the ability to easily call interpreted L# at runtime will still have its place in many other scenarios.

Tags: .Net, L#, LSharp, C#, Lisp

5 Reasons Why Joel's Business (Probably) Isn't Like Yours

Joel Spolsky has had a lot of interesting things to say about software and the software business, including this week when he had a whole series on hiring. In this week's series, he said lots of things that programmers are bound to like: treat your developers like rockstars, give them all Aeron chairs and super-expensive monitors, build an elaborate atrium with a fountain just to impress your developers.

You have to admire Joel's approach, because it says right on his site that his business is all about programmers:

Best Working Conditions

=>

Best Programmers

=>

Best Software

=>

Profit!

If you're a programmer, how can you not like that?

I'm sure his recent series (and maybe his blog in general) has left a lot of people thinking: wow, my company isn't like that. Joel is so much cooler than my company. And, well, maybe he is. But before you go in to your boss and start complaining about how he's cheap, and if he would just run his business like Fog Creek, things would be so much better, here are a few reasons to take his ideas with a grain of salt and think about whether they would really benefit the company where you work.

By the way, I'm not really an expert on Fog Creek or anything, so I'm inferring a lot of this as I go along. And I'm not doing this to trash Joel, because I love reading his writing, so if you're reading this to get to the good parts where I really rip him a new one, well, sorry.

1. Fog Creek is a software business

In a software business, the developers drive the products. Programmers are always going to be pretty vital to a software company. By contrast, at an automaker or a bond trading firm, it's going to be people like car designers or bond traders that are the rockstars -- they're the ones that make the business go.

By the way, here's a corollary: all other things being equal, it's more fun for a programmer to work at a software company than to write software for a business in another industry. Any software company that cares about its products cares about its programmers.

2. Joel doesn't have 2000 people

This is the "not everyone can be a rockstar" rule. The larger a company gets, the harder it is to only hire the kind of superstars Joel talks about looking for. It's not clear that his model scales to bigger businesses.

Then again, he learned the software business at Microsoft which always used to have a rep for only hiring brilliant people, so maybe it's not impossible to scale this model, but merely really difficult.

3. Joel started up with a good-sized chunk of capital

This is just speculation -- I could be wrong. But his basic premise of starting a really cool business with Aeron chairs and an atrium and expensive monitors makes it sound like he (and maybe his occasionally-mentioned business partner, who I don't know anything about) had a bunch of Microsoft options that vested and brought in a bunch of money as seed capital.

I don't mean that in a dismissive way, such as "well, anyone could start a successful software business if they had that kind of capital." That's actually not at all true. Most Microsoft people who vested and started their own businesses completely failed. What it means is, if you start a business by maxing out a Visa card, for god's sake, don't spend the money on an Aeron chair.

Also, because I presume that the capital was mostly his, he didn't have people pressuring him to show a large immediate return. If you have Venture Capitalists breathing down your neck, don't go hook yourself up with an office with a fancy atrium, just because Joel told you to. Keep your burn rate low, like everyone learned from the dot com crash.

4. Fog Creek makes shrinkwrap software

Joel wrote a really good essay a few years ago about the 5 Worlds of Software Development. One of them is shrinkwrap software, which is what Fog Creek does.

Shrinkwrap software is great. Everyone actually pays you to use your software, which, if you do it right, makes it a lot easier to pay your bills than if you write open source and hope to live off support contracts or build a cool website and hope to live off Google AdWords.

Not only do you get paid up front, but the best part about shrinkwrap is that it scales like crazy. If you work in an IT department and build internal software, you work closely with your client, and you usually can't easily add any more clients. If you write software with a heavy customization component, you pretty much have to add new fleets of consultants or VARs as fast as you add clients. But if you sell shrinkwrap, then as you add more clients, the money from the new sales mostly goes straight to the bottom line.

What this means is that if you have a shrinkwrap company, you have the best possible situation to actually generate a return on hiring rockstar programmers and throwing expensive perks at them. You have a situation where you get paid for producing software people want, and there's a significant asymmetric relationship between the work that goes in and the money that comes back if you're successful. The only thing you have to watch out for is secondary costs like technical support, which merely reinforces that you have to build and test it right in the first place.

Here's the thing, though: as evidenced by the fact that there aren't many startups doing shrinkwrap anymore, it's a tough business to succeed at. As you would expect from the asymmetric returns, and as evidenced by Microsoft, it tends to be a winner-take-all business. More sales => more money to invest in products and marketing => even more sales. So what does it take to be successful? Well, one is good products up front, so you can sell some in the first place. This is where the whole rockstar programmers thing comes in, and why it makes sense for Joel to give everyone Aeron chairs.

The other big one, of course, is simple market awareness: get your product in front of people. And that's where the blog with millions of hits comes in. Which brings us to my last point:

5. Joel's target market is programmers

If you've ever heard MBAs wet themselves talking about synergy and thought that it was all a pile of B.S., well, sometimes even MBAs are right. Because Joel's business is just one giant perpetual motion machine of synergy. He clearly spends a ton of time writing blog entries about software development and software businesses, but it all pays off, because his target market is other programmers. Soccer moms and MBAs surely don't have any use for bug tracking software and remote-support applications -- developers do. So Joel's investment of his time in his blog pays off in massive brand awareness among his target market.

Joel writes all these things about hiring, and how programmers should be treated as rockstars. And one benefit is that he must have one monster applicant pool. But another is that it ties right into what his target market wants to hear. Not only that, but after all the time he spends talking about how he only hires the best of the best, who wouldn't want to at least evaluate his software if they need bug tracking? I mean, what kind of schmuck would settle for ordinary software when they can have code written by supergeniuses (even if it is in VBScript)?

Now, someone is going to read this last point and think I'm saying that Joel is evil: he's just doing all of this to trick us into buying his software. But that's not my point at all. The guy has built a business where he gets to go to work in a fun environment and write code with lots of other smart people. He gets to be famous in the blogosphere, and have lots of people hanging on his every word. And as icing on the cake, he gets to profit from the whole thing on top of it. There ain't nothing wrong with that other than that we can't all be that lucky (though really, it isn't luck -- it's a smart business plan).

But like I said at the start, your business (probably) isn't like that. So if you don't have the Aeron chair and the fountain and the 30" monitor, it probably has something to do with one of these reasons.

A Simple Library of L# Utility Functions

As I've been playing around with L#, I ran across a few pretty universally useful things that don't ship with the release. So I whipped up a basic library that contains a few useful things. I should have a permanent home for this soon, but I don't at the moment, so I'm just going to post it here for now.

I'm far from a Lisp expert, so if anyone wants to work out a better implementation of some of this code, I'll be happy to include it in the next version.

; basics.ls -- implements some universally used functions in
;              the L# language
;
; modified 9/7/06: replaced references to nil with null,
;                  simplified filter, map, and reduce,
;                  added get function.
;
; Copyright (c) 2006, Andrew Norris
;
; This file doesn't contains anything especially remarkable,
; and anyone is free to reuse this in whatever fashion you
; like, and include it in a product licensed under any terms.
; Please retain the copyright attribution. Thanks.

; reasonably efficient list concatenation function
(= concat (fn (&rest items)
    (let sb (new system.text.stringbuilder)
      (foreach elem items
        (call append sb elem))
      (tostring sb ))))

; recovers the string value of an object for use in code
(= str (fn (obj)
    (writetostring printer obj)))

; applies function f to a list of items and returns the
; result
(= map (fn (f lst)
    (if (== lst null)
      null
      (cons (f (car lst)) (map f (cdr lst))))))

; returns the items in the list for which f(item) is true
(= filter (fn (f lst)
    (if (== lst null)
      null
      (if (f (car lst))
        (cons (car lst) (filter f (cdr lst)))
        (filter f (cdr lst))))))

; returns the result of applying a group function over
; the list lst
;
; for example, (reduce (fn (a b) (* a b)) '(2 3 4 5) 1)
; will initialize with the value 1, and return the
; product of all of the values of the list, i.e.
; 1 * 2 * 3 * 4 * 5 = 120
(= reduce (fn (f lst init)
    (if (== lst null)
      init
      (f (car lst) (reduce f (cdr lst) init)))))

; pushes item on to the front of list lst, altering the
; actual list passed in
(defmacro push (lst item)
  `(= ,lst (cons ,item ,lst)))

; pushes item on to list lst, altering the actual list
; passed in
; note: uses a goofy variable, because 1.2.1 doesn't have
; gensym
(defmacro pop (lst)
  `(let obscure-item-name (car ,lst)
    (= ,lst (cdr ,lst))
    obscure-item-name)))

; traverses a property list and finds a matching symbol
(= get (fn (plist sym)
    (if (== plist null)
      null
      (if (eq (car plist) sym)
        (cadr plist)
        (get (cddr plist) sym)))))

Update: I fixed a couple of bugs, made the code a little more elegant, and added the get function. I don't have a permanent home for this yet, so I just applied the changes here.

Tags: L#, LSharp, .Net, Lisp, code, library

First Impressions of L#

Introduction

Lately I've been fooling around with Lisps. I've done some work on Linux in SBCL, an open source Common Lisp implementation that seems like a good, solid piece of code. But I've spent most of my time writing code in L#, an interpreted Lisp dialect for the .Net CLR. Lots of people are familiar with Common Lisp, but not all that many people seem to be familiar with L# yet, so I thought I would share my first impressions.

Like apparently everyone else who has started learning Lisp in the last five years, I got interested by reading Paul Graham. In particular, I didn't like the idea of programming in Blub without knowing more about some of the available alternatives.

So I've read Peter Seibel's Practical Common Lisp and I've started reading Paul Graham's On Lisp, and trying to learn how Lisp works.

Of course, if you keep reading Paul Graham, you also notice that he doesn't exactly endorse Common Lisp. He's been working on a newer, sexier Lisp called Arc, which of course you can't use yet. In particular, he argues the importance of having powerful, modern libraries.

It was somewhere around then that I ran across L#. L# was designed by Rob Blackwell with a lot of Graham's published ideas for Arc in mind. To me, at least, the language looks a little cleaner. In addition, it's designed to run on top of .Net, which means that even in its early stages, it comes with complete with a massive set of libraries for everything from XML to database access to user interfaces. When you add in the fact that I'm employed as a C# developer, so I already know those libraries pretty well, L# seemed like a no brainer to try out. So naturally, I did.

There's a lot to like about L#

If you're still with me at all, you're probably ready for me to move past the preamble and actually talk about L#. So here we go.

The single best thing about L# is exactly what you would expect: it combines a Lispy (and very clean) syntax with the massive library support of .Net. Want to do regular expressions? There's a .Net library that does all the heavy lifting. Hate all the gratuitous verbosity you get in .Net? Write a 5-line wrapper function and go to town.

Here's a really simple but real-life example: if L# comes with a concatenation operator, I haven't found it yet. I was doing a whole lot of string concatenation operations, and wanted an easy way to do it. In C# there are two basic ways to do string concatenation. The expensive way with clean syntax is to do:

mystring = foo + bar + baz + "\n";

which instantiates a new string for each concatenation operation. Meanwhile the efficent way with heinous syntax is to use a StringBuilder, which has a buffer so you don't have to reallocate space every time:

StringBuilder sb = new StringBuilder();
sb.Append(foo);
sb.Append(bar);
sb.Append(baz);
sb.Append("\n");
mystring = sb.ToString();

You could encapsulate a function for doing this, of course, and end up with something like:

string[] myarray = { foo, bar, baz, "\n" };
string mystring = MyStringLibrary.Concatenate(myarray);

That's a little better, but it's still kind of an eyesore. You can't declare the array inline as an argument to the function, and you have to carry around a class to hold the function name.

It won't surprise anyone familiar with Lisp, but L# makes it easy to make it easy. Here's my simple function to concatenate a list. I expect a Lisp guru could do something cooler, but I'm nowhere near a Lisp guru yet:

(= concat (fn (&rest members)
    (let sb (new system.text.stringbuilder)
      (foreach elem members
        (call append sb elem))
      (tostring sb))))

This yields a nice clean syntax for string concatenation, with all the advantages of the efficient, ugly way in C#, and a syntax as simple as adding strings together:

(concat foo bar baz "\n")

Actually, I seem to be using string concatenation enough that I think I'm going to rename the function ct, which should make things even more terse.

So anyway, that's just the simplest, scratch-the-surface example. Once you get into macros and things like that, I'm pretty sure that the differences will be even more dramatic.

L# shows enormous potential as a glue language for .Net applications: scripting, configuration, things like that. For scripting tasks, you can easily write simple, terse scripts that work with the code you've already built. I feel a little silly recommending a variant of possibly the most powerful programming languages ever devised as a way of scripting something written in C#, but it's a good way to get a foot in the door and add L# to your toolchain.

Steve Yegge has made a pretty compelling case for Lisp as a replacement for XML in things like config files and logs. I don't have much to add to that, but if you've ever gotten tired of messing around with XSLT transforms, L# offers an easy way to abandon them forever.

So utility programming in L# is pretty cool, but can you do more than that? Can you write real apps? I don't know yet. But I plan to find out. A language that's really powerful, but still fundamentally easy and fun to work with is worth experimenting with to find out how far you can take it.

I'm sure I won't be able to rip out years of production code and replace them all with L# -- really, I wouldn't want to, even if I could. But there's no reason not to look at it for new projects and interoperation. Currently, there's a few missing pieces for seamless interoperation with other .Net languages (mainly, L# doesn't compile into callable assemblies yet), but there's nothing standing in the way of getting those problems solved except time and effort.

L# Tools

The current tools for developing in L# aren't everything you could want, but they're not bad. First off, you can use Visual Studio, but there are no direct integration tools yet. Even without syntax highlighting, etc., this is still a pretty good solution. The main reason it works pretty well, is that it gives you the ability to debug your code. Well, sort of.

Actually, what you can do is load the C# source code for the LSharp interpreter into Visual Studio and debug the interpreter. You set breakpoints in places like Eval() and Call() and follow your code as it evaluates. This took a little adjusting to, but it actually works pretty well. You can drill down and see exactly what it's evaluating and how it's failing. Did you forget to quote something and send an unquoted symbol by mistake? It's all right there in the interpreter -- you can see what it's doing with your code.

Outside the debugger, the main advantage of using Visual Studio is that if you're integrating it with C# in the same solution, you can manage all the files in the same place. But really, to Visual Studio your L# files are just opaque text files.

Another tool I've worked with is TextPad, which is a feature-rich text editor for Windows that I use anyway. There's a syntax file for L# for TextPad, so it gives you syntax highlighting and parentheses matching (with ctrl-M). I often edit L# files in TextPad at the same time that I have the project open in Visual Studio, and Visual Studio has a handy reload dialog that pops up when files have changed.

There are other tools for working in C#, but I haven't had a chance to put them through their paces yet. xacc.ide is an integrated development environment being built that not only has support for L# syntax, it apparently uses L# as a scripting language. That certainly implies that xacc.ide will be a good tool for L# development. Finally, emacs support for L# has been developed, so if you're comfortable working in emacs, that may be just the ticket.

Conclusion

Of course, not everything in L# is ideal. I've run across some annoyances as I've been working in L#, as you would imagine for a young language. Nothing that is a showstopper, but a few things I wish worked a little better. The main thing, though, is that I haven't run into anything that has made me even want to consider abandoning L# for another language.

I'm hopeful that at least most of the annoyances can simply be fixed, so I'm going to write a followup post that goes into these in more detail, along with some ideas about what to do about them. This post is long enough as it is.

In the meantime, I hope this helps people know more about what to expect with L#, and I hope that people will go ahead and try it out. It's a solid language, and it potentially adds a whole new dimension to .Net projects.

Tags: .Net, L#, LSharp, Lisp, C#