A problem with the UNIX CLI shell

The UNIX shell is an amazing tool to boost your productivity. The concepts are quite simple – the shell allows you to type commands that it executes in the order you provide – there is the possibility of chaining commands so that the output of one command to be the input for another one. You can create loops and have a simple yet effective programming language available.

Unfortunately, the UNIX shell is obsolete. It’s a concept conceived in the early 1970s and unchanged since. There are now completion options (you can autocomplete stuff with the TAB key) but that’s about it. There were not many changes in the last three decades, and, if we discard the bug that scared the whole internet last year, there is little talk about the command line shell.

However, we should talk. There are conceptual mistakes in there that make the shell insecure and unsafe to use. I will not enumerate them all here, but I want to start with an idea that bothers me greatly – the deployment of packages in the Linux systems.

I love RPM. RPM is great, even better (for me) than dpkg because I know RPM – I have my share of spec files written, as I built my own RPM-based distribution. However, I was always upset about the fact that everything has to follow the /usr/{bin|lib|etc} folder structure. I feel this is counter-intuitive. and useless. I do understand the desire to separate executables from the libraries, but I feel that the concern works against users – and, even worse, forces them to install things in the “system” if they want upgrades over what the system offers.

Definitely, packages can be installed in a root folder under the home folder of a user. It CAN be done, it doesn’t mean it should be done – and people don’t do it often because it’s not handy. It’s not handy to have a /opt folder that does things for you, because you have to set your PATH and LD_LIBRARY_PATH variables and a lot of other things. These variables should be eliminated from the user’s view, as they are also huge security breaches1)I will not explain further, but, in a few words, think of an executable called “ls” placed in your PATH and that does the job of rm for the current folder.

A secondary problem is that the existence of the PATH and friends turns flat a file system hierarchy, and basically eliminates any performance advantage one might get from the tree structure (faster search times)

So what I would like instead is to have all main packages installed in their own folder. Do I want to call python? All I need to type is python.run – this will search for the “run” command in the python package. And that package is in its own folder – it will run as if it has the PATH set to “/package/python/bin;/system/bin”, LD_LIBRARY_PATH=”/package/python/lib:/system/lib” and so on. The executable will have a clean, isolated context that it will protect it from intrusion – as an administrator all you need to preserve is the file system – you don’t want to be bothered with system variables. One administrative task less.

And you won’t have to worry that your packages want to offer the same executable. You want IcedTea and Java from Oracle? Ok! You can differentiate by using the commands icedtea.java vs. java.java (for example).

This is just a starting idea. It can be developed further. Maybe I’ll continue with this train of thoughts.

ComputerProgramming

There are 98% chances in the future that cup of coffee will spill over the keyboard. (foto via pixabay)

NOTES   [ + ]

1. I will not explain further, but, in a few words, think of an executable called “ls” placed in your PATH and that does the job of rm for the current folder

Comments

A problem with the UNIX CLI shell — 25 Comments

  1. Your complaints seem to be mostly about the package managers and less about the shell itself.

    A secondary problem is that the existence of the PATH and friends turns flat a file system hierarchy, and basically eliminates any performance advantage one might get from the tree structure (faster search times)

    Just… how long is your $PATH, man? Linux distros are moving towards putting everything in /usr (the so-called /usr unification), so at some point you’ll have all the executables in /usr/bin and all the libraries in /usr/lib. But a few extra lookups were not much of a performance problem 15 years ago when we were using high latency spinning disks and they’re becoming even less of a problem every day. We have SSDs on PCIe now.

    But anyway, Linux is my main operating system and I honestly don’t remember the last time I had to deal with environment variables. If you find you constantly “want upgrades over what the system offers” consider switching from CentOS to Fedora. 😀

    So what I would like instead is to have all main packages installed in their own folder.

    Self-contained packages would be nice, but you’ll probably want to have the libraries separated from the executables anyway. Otherwise you’ll have to trust all the http clients on your system (browsers, curl, wget etc) to upgrade their libssl when there’s a problem with it. This is also one of the problems with containers, although Docker (sort of) solves it with image composition. So if you want to have an easily upgradable system-wide libssl you’ll probably end up with some sort of /usr/bin vs /usr/lib separation anyway.

  2. This is something which worth a new linux distribution.
    Adecatelea decât sa faci un Ion (sau cum le zicea la distributiile alea românești) doar pentru că are alt nume mai bine faci o distributie nouă pentru că vrei să demonstrezi un concept de genul ăsta.

    • I want to refine the idea, then express it in a new, small distro. Probably it will not include the X server, but there’s plenty of time for such decisions in the future. Right now I’m only interested in a shell. 🙂

  3. I’m sory, Dorin, but you sound like a beginner in the Unix field ! You’re disappointing me !
    First, the Unix filesystem structure should be followed for PORTABILITY reasons ! Shell and other packages should be portable also to other Unix flavors (BSD, etc), not only Linux ! Why do you want to change that ?
    The environment variables are also used in the ELF structure, not only by shells, removing them breaks everything ! Have you considered that ?
    For your requirements, it is enough to define a icedtea.java link in /usr/bin to java app starting script, where to define the environment variables required to run your java app flavor (no need to inform the user about these variables).
    This solution is already a used by chromium or google-chrome, as an example (accepted by wise thinkers from Google, you have a problem with them ?). No need to ask users to know anything about paths.

    • I want to change that because 1) I’m not interested in filesystem level portability. 2) Breaking everything is how one starts to fix issues. 3) If you’re not ready to think different, you’ll use magic hammer solutions forever.

      And I think that Linux is the right solution, the problem is the GNU system that sucks.

    • Fixing issues by breaking everything, is not wise. I also had these thoughts when I was a beginner, but now I can fix anything by just using the actual system options (by knowing everything about them, in details). Call it experience.

    • It was equally unwise to start a kernel project in 1991.
      Not that it compares. But heck, why not? SysV is obsolete why not have something new? Aren’t you bored with GNU?

    • Yes, but Linus was backed by GNU people. Linux would be dead, without GNU. There is a GNU alternative, uClibc and busybox. But it would be dead, too, without borrowing GNU principles (basic libc definitions).
      Who do you think would backup you ? There are thousands of distributions which died, due to lack of backup. You need a strong reason to be helped by anyone. And also a lot of experience. Not talking about free time.
      It sounds like you want to invent a new car, just because you don’t like the interior of the current ones.

    • I can’t be constructive without a good reason. From your post, your reasons sucks (because they can be fixed without breaking anything). Please came with some really good ideas, and I may help you.
      Otherwise, it would not worth it. Just waste of time.

    • Omule, hai să-ți explic pe românește, că nu pricepi. Nu ești binevenit aici. De când ai venit să-mi explici că tu mă știi și deci ești cu trei nivele peste mine în materie de opinii ai fost rugat să faci pași până în momentul în care înveți să te prezinți ca o persoană normală. Ești nesimțit și ești unul din motivele pentru care îmi închid cealaltă parte a blogului. Genul ăsta de nesimțire, lipsă de respect față de interlocutor, îmi repugnă. Ceea ce faci tu e josnic. Chiar vrei să începem să discutăm în termenii ăștia? Chiar trebuie să-ți explic ca la copiii cretini de grădiniță cum trebuie să te comporți în societate? Pentru ultima. INSIST, pentru ULTIMA dată îți las șansa la replică. Dar prefer să dispari de pe blogul meu. Nu vreau să sufere ceilalți cititori din cauza ta.

    • Omule, ideea de dialog e ca să dezbatem o temă, să vii cu opinii care prin care să-mi demonstrezi că greşesc în ceea ce am spus ! Dar tu nu vrei să ţii un dialog, du doar le iei personal şi te superi ! Nu prea iţi place să-ţi argumentezi ideile, din câte văd. Pentru că dacă le-ai argumenta, le-ai stăpâni mai bine şi ai fi mai convins de ele (căutqnd argumente, observi detaliile care fac diferenţa). Examinând argumentele, aceastea se adună la experienţa ta (un plus pentru tine).
      Eu îţi spuneam doar că un proiect de nouă distribuţie nu merită, e o pierdere de timp. Te aşteptam cu argumente care să-mi dovedească contrariul (demolându-mi argumentele), nu să o iei personal. Intr-o distribuţie sunt atât de multe de făcut încât nu o să ai timp să-l termini vreodată, să-l aduci la o formă rotundă, ceea ce duce la insatisfacţii enorme. Ceea ce nu cred că vrei.

    • Dacă vrei să avem un dialog te prezinți frumos, ca să știe toată lumea cu cine vorbim. Altfel, opiniile tale sunt insignifiante. “Anonimatul” îți asigură numai banca chibiților, atât, nimic mai mult. Refuz să îți mai accept comentariile sub protecția anonimatului.

    • ești unul din motivele pentru care îmi închid cealaltă parte a blogului

      Wow… Mă faci să mă întreb dacă n-oi fi și eu printre celelalte motive. 😀

  4. Yeah, but you see, the problem is that when one has

    When one has what? If you’re a very atypical user you can’t blame it on general-purpose Linux distributions that they don’t cater to your very specific needs. Well, you can, but it’s not reasonable.

    • I never said change is generally unreasonable. I just took your “the problem is that when one has” (?!) to mean “I have special requirements”, and I said that if you have special requirements it’s unreasonable to state that a general-purpose distribution has a problem for not meeting them.

      Suppose, for instance, that you had no legs and you stated that cars had a problem because they have pedals. No, cars in general don’t have a problem, you just need a customized one.

    • Btw, if you need inspiration there are some fringe distributions out there like GoboLinux (mostly known for their attempt to reorganize the Linux filesystem) and a new and growing class of distributions that use containers for transactional, self-contained software packages (CoreOS, Snappy Ubuntu etc). The latter are currently pretty server-oriented.

  5. “There are conceptual mistakes in there that make the shell insecure and unsafe to use. I will not enumerate them all here, but I want to start with an idea that bothers me greatly – the deployment of packages in the Linux systems.”

    You should change the title. A suggestion would be “Why the *nix filesystem architecture is not good for me”. Then, after some time you will realize they had some serious reasons for doing this separation when installing an application (which means one or more packages)

    And, unfortunately for you, there is a standard ( https://wiki.linuxfoundation.org/en/FHS ) for that which will probably piss you off since you know that standards tend to be implemented when portability is required.

    “A secondary problem is that the existence of the PATH and friends turns flat a file system hierarchy, and basically eliminates any performance advantage one might get from the tree structure (faster search times)”

    You want to compare log(n) complexity (not actually that fast as it is not a binary tree) with O(1) complexity (a finite list of paths whose executable list might easily be cached – in my case “/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin”)?

    “The executable will have a clean, isolated context that it will protect it from intrusion – as an administrator all you need to preserve is the file system – you don’t want to be bothered with system variables. One administrative task less.”

    I don’t see how this adds a protective layer to an installed software. Maybe this needs to be further elaborated. I guess that if your machine is hijacked then things are the same as with current distributions. But as I said I may not fully understand this. I don’t see an advantage to the sysadmin because all he must do is to reinstall the (updated) package(s).

    “And you won’t have to worry that your packages want to offer the same executable. You want IcedTea and Java from Oracle? Ok! You can differentiate by using the commands icedtea.java vs. java.java (for example).”

    This is actually solved in some distros with “update-alternatives” or similar tools. But setting a path in a profile script (depending on your shell) didn’t hurt anyone to run jdk, android or other weird tools as long as they were executed with restricted rights.

    In conclusion I wouldn’t start on such path because:

    I don’t think there is an actual problem here (if there is it needs to be proved … with facts)
    Creating a (new) specification is not easy
    2.1 Making it a standard is hell
    I cannot convince people that I can do better without any serious funding or backup (but then again, you might be luckier than most people)
    I consider that only companies with big pockets can make this happen but even so they don’t give much shit since this works
    A serious problem with talented people is that they do useless things because this is what they feel. I did this several times and I hope to do it less in the future, even if that means less “fun” for me. Tech people need to be remembered that as long as they are not independent (financially) they shouldn’t pursue paths that don’t lead to any tangible profit ( unless they are paid to do so 🙂 )

    • Sorry for replying so late. It seems that Akismet was eager to mark your message as spam. So:

      1) I will not change the title – the title is not that important, I should’ve done it in the first place, but then I got distracted. There’s little value in changing the title, which is still valid, from my point of view.

      2) I know FHS. I know FHS and actually read the whole standard – so many decisions are made so poorly you can see it clearly in, for example, the need to have special X11 folders. There’s too much clutter and the FHS acknowledges that – everytime you see an exception, there’s the FHS not being able to handle change. And that’s my point – change is needed. Portability is less of an issue than you think. Portability is not achieved by hardcoding /etc/X11/whatever, but instead really having something like ‘GetConfiguration(“X11”, “whatever”)’. A unifying API is needed, that would ensure portability.

      3) My proposal is O(1), your proposal is O(n) – no matter what sort of caching you do.

      4) Think chroot. Easier to do when everything is isolated.

      5) update-alternatives is a cumbersome solution to the problem. It tries to manage the unmanageable. It kind of fails from my perspective, and it’s still hard to use both Java systems at the same time.

      6) You don’t have to start on my path.

      7) I will not explain the obvious. The problem is there, you’re just used to things the way they are.

      8) Who says I want to create specifications? I am tempted, but it’s not my main objective.

      9) Perhaps I’m not talented, I’m only a hot-head with an idea. It’s quite more likely.

Comentariul tău