If you are interested in a shell with first-class function, list and map values and pipelines that pass such complex data (instead of structureless bytes), you might be interested in elvish: https://github.com/elves/elvish
If there is a better summary, let me know, or feel free to edit the wiki page yourself. (I think it should be editable.)
I've been in contact with the authors of the oh shell, NGS shell, and mash shell, and exchanged some interesting ideas. The design of the oil language should be more concrete in 2-3 months and I'm hoping to get feedback from others who have thought about this space. I might ping you then :)
Are the elvish structured data pipelines actually separate processes, or are they goroutines? Is there a format for piping them over say SSH? I have been thinking about this a little bit for oil.
Hi chubot, I didn't realize that you are developing a shell as well. It's great to see competitions. Can you share some ideas that you learned from the authors of said shells? I would love to hear those, designing shells is not an easy task.
Structured data pipelines run in the same process as elvish, in goroutines. For interacting with external commands you need to serialize and deserialize, like this:
~> put [foo bar lorem ipsum] | to-json |
python3 -c 'import sys, json;print(json.dumps([x.upper() for x in json.load(sys.stdin)]))' |
from-json
▶ [FOO BAR LOREM IPSUM]
I had some good discussions with Michael MacInnis and recently Ilya Sher. I had actually read Michael's masters thesis in 2011 or 2012 and found it to be a very good historical overview of shells:
I agree with the goal of "taking shell seriously as a programming language", which I think all projects share. I disagreed on the need to make shell a variant of Lisp. I actually tried this a bit -- I hacked on femtolisp, which is used to bootstrap Julia, and I thought I could bootstrap a good shell with it. This didn't work out as nicely as I had hoped so I dropped the idea.
Ilya was having some issues with the "two modes" of parsing -- command mode for shell-like constructs and expression mode for programming features (function calls, math, etc.). I thought this would be an issue too, but as mentioned in another comment, now that I've written a bash parser I think it's straightforward to solve, and Ilya agreed.
The thing is that bash already has many different modes of parsing, which I implement using "lexer modes" or lexical state. This same technique can be applied to a new shell and then there should be a natural way of mixing commands and expressions (quoted and unquoted words, etc.):
I'm also trying to make oil do what bash/dash/etc. do essentially syscall for syscall. I believe that is a little hard with Go because it has its own runtime. Although I think there is a way to pin a goroutine to a single thread which can mitigate those issues.
I was planning to start a thread in maybe a month to review some of these issues and exchange feedback. Let me know if you are interested!
Disclaimer: I am the author of elvish.