Yeah, "path dependence" pretty much sums up the comment. I should probably write something about that, because a huge factor in the design of https://oilshell.org. But I still get questions about it (like "Why can't we just get rid of shell and start over from scratch?")
A few points I would like to add on:
- I don't view the existence of TypeScript, MyPy, and Sorbet as evidence in favor of static typing. It's evidence in favor of gradual typing!
- The Oil project gave me a lot of experience with the relation between metaprogramming/reflection, dynamic types vs. static types, performance, and code length.
In particular the code moved from dynamic to static typing, and got a lot faster. But dynamic typing wasn't a mistake.
Short recap: Oil's code is 5-7x shorter than bash [1], and a lot of that is due to starting out as dynamically typed, with a lot of metaprogramming. (I still believe "size is code's worst enemy" -- big code is understood more poorly, which makes it harder to modify, regardless of static types.)
It is also something like 30x-50x slower than bash in Python! So, unusably slow. However the surprise is that I statically typed this code, and semi-automatically translated it to C++, and the result is now faster than bash. [2]
-----
So the high level, short code has enough semantic information to be fast (after you add explicit types).
The static typing process mainly evolved expanding metaprogramming to textual code generation! I estimate that this was at least 9 months of rewriting.
That's what your arguments are missing IMO. If you're writing Java in Python, then sure Python is going to seem like it offers no advantages, and it might as well be statically typed.
But that's not how people write programs in dynamic languages (and honestly I thought you would appreciate that more, having written so much about dynamic languages!) The porting process taught me exactly how much dynamism I was using, and it was a lot! It was pulling a lot of weight.
I should show all the code generators and generated code in an essay... it's a very concrete demonstration.
So there is the fallacy of "type inference" solves the problem -- it's not that we're too lazy to write down types; it's that we're using techniques that static type systems can't handle. Good thread about that: https://twitter.com/sliminality/status/1317331149354463232
I'm not saying the Oil experience generalizes, since it's an unusual project, but it's definitely not as simple with "go with static types so you don't get trapped". That said, the conundrum you're talking about is very real.
-----
But despite writing all that, I'm actually leaning in your direction, and I started a statically typed language :)
I would start with an interpreter so it can have metaprogramming (e.g. like a constexpr interpreter, or what Zig does). I would like to add a gradual type system, but I don't really know how to write one, so the first cut will be a traditional static type system. (In this world, getting the 30-50x speedup relies on the program being 100% statically typed, yet gradual typing is still important IMO. It's very simple, no infinite treadmill of JIT work as you see in the "professional" projects.)
-----
This is probably something for an essay, but I would say dynamic types are demonstrably better than static types for at least 3 domains: UI, data science, and security/reverse engineering (and I have a bunch of experience to back this up). Basically anything that involves "learning about the world", or "schema discovery".
Really short summary: I used the MyPy front end, and basically printed its AST as C++. This involves a bunch of hacks, but it works.
The following conditions helped a lot:
- I control all the code in Oil, so I can statically type all of it, which involves both annotations and occasional patches. A shell doesn't have many Python library dependencies; i.e. it doesn't depend on BeautifulSoup or something like that. It's basically all string and data structure manipulation, with a few sys calls.
- Oil has extensive tests. I run the same tests against the translated and compiled C++, which flushes out bugs in the translator. (Something like 915 out of 1700 tests pass now, so the translation process isn't done.)
- I also used unit tests to generate some of the type annotations with pyannotate.
There is still some translation left to do if anyone is interested in helping. You will probably learn something about both Python and C++!
A few points I would like to add on:
- I don't view the existence of TypeScript, MyPy, and Sorbet as evidence in favor of static typing. It's evidence in favor of gradual typing!
- The Oil project gave me a lot of experience with the relation between metaprogramming/reflection, dynamic types vs. static types, performance, and code length.
In particular the code moved from dynamic to static typing, and got a lot faster. But dynamic typing wasn't a mistake.
Short recap: Oil's code is 5-7x shorter than bash [1], and a lot of that is due to starting out as dynamically typed, with a lot of metaprogramming. (I still believe "size is code's worst enemy" -- big code is understood more poorly, which makes it harder to modify, regardless of static types.)
It is also something like 30x-50x slower than bash in Python! So, unusably slow. However the surprise is that I statically typed this code, and semi-automatically translated it to C++, and the result is now faster than bash. [2]
-----
So the high level, short code has enough semantic information to be fast (after you add explicit types).
The static typing process mainly evolved expanding metaprogramming to textual code generation! I estimate that this was at least 9 months of rewriting.
That's what your arguments are missing IMO. If you're writing Java in Python, then sure Python is going to seem like it offers no advantages, and it might as well be statically typed.
But that's not how people write programs in dynamic languages (and honestly I thought you would appreciate that more, having written so much about dynamic languages!) The porting process taught me exactly how much dynamism I was using, and it was a lot! It was pulling a lot of weight.
I should show all the code generators and generated code in an essay... it's a very concrete demonstration.
So there is the fallacy of "type inference" solves the problem -- it's not that we're too lazy to write down types; it's that we're using techniques that static type systems can't handle. Good thread about that: https://twitter.com/sliminality/status/1317331149354463232
I'm not saying the Oil experience generalizes, since it's an unusual project, but it's definitely not as simple with "go with static types so you don't get trapped". That said, the conundrum you're talking about is very real.
-----
But despite writing all that, I'm actually leaning in your direction, and I started a statically typed language :)
https://old.reddit.com/r/ProgrammingLanguages/comments/jb5i5...
I would start with an interpreter so it can have metaprogramming (e.g. like a constexpr interpreter, or what Zig does). I would like to add a gradual type system, but I don't really know how to write one, so the first cut will be a traditional static type system. (In this world, getting the 30-50x speedup relies on the program being 100% statically typed, yet gradual typing is still important IMO. It's very simple, no infinite treadmill of JIT work as you see in the "professional" projects.)
-----
This is probably something for an essay, but I would say dynamic types are demonstrably better than static types for at least 3 domains: UI, data science, and security/reverse engineering (and I have a bunch of experience to back this up). Basically anything that involves "learning about the world", or "schema discovery".
[1] http://www.oilshell.org/blog/2019/06/17.html#why-is-it-writt...
[2] https://www.oilshell.org/blog/2020/01/parser-benchmarks.html
Good thread about how successful Ruby and Python have been in YC companies: https://news.ycombinator.com/item?id=24279611