diff options
Diffstat (limited to 'blog/dependency-hell-2014-11-20.markdown')
| -rw-r--r-- | blog/dependency-hell-2014-11-20.markdown | 105 |
1 files changed, 105 insertions, 0 deletions
diff --git a/blog/dependency-hell-2014-11-20.markdown b/blog/dependency-hell-2014-11-20.markdown new file mode 100644 index 0000000..f9eab55 --- /dev/null +++ b/blog/dependency-hell-2014-11-20.markdown @@ -0,0 +1,105 @@ +--- +title: Dependency Hell +date: 2014-11-20 +--- + +Dependency Hell +=============== + +A lot of the problem that I have run into when doing development with +nearly any stack I have used is dependency management. This relatively +simple-looking problem just becomes such an evil, evil thing to tackle. +There are several schools of thought to this. The first is that +dependencies need to be frozen the second you ever see them and are only +upgraded once in a blue moon when upstream introduces a feature you need +or has a CVE released. The second is to have competent maintainers +upstream that follow things like semantic versioning. + +### Ruby + +Let's take a look at how the Ruby community solves this problem. + +One job I had made us need to install **five** versions of the Ruby +interpreter in order to be compatible with all the different projects +they wrote. To manage the five versions of the Ruby interpreter, they +suggested using a widely known tool called +[rbenv](https://github.com/sstephenson/rbenv). + +This isn't actually the full list of rubies that job required. I have +decided not to reveal that out of interest of privacy as well as the +fact that even Gentoo did not ship a version of gcc old enough to build +the oldest ruby. + +After all this, of course, all the dependencies are locked using the gem +tool and another helper called bundler. It's just a mess. + +There are also language design features of ruby that really do not help +with this all that just make simple things like "will this code run or +not" be determined at runtime. To be fair, Python is the same way, as is +nearly every other scripting language. In the case of Lua this is +*beyond vital* because of the fact that Lua is designed to be embedded +into pretty much anything, with arbitrary globals being set willy-nilly. +Consequently this is why you can't make an autocomplete for lua without +executing the code in its preferred environment (unless you really just +guess based on the requires and other files present in the directory). + +### Python + +The Python community has largely copied the ruby pattern for this, but +they advocate creating local, project-specific prefixes with all of the +packages/eggs you installed and a list of them instead of compiling an +entire Python interpreter per project. With the Python 2-\>3 change a +lot of things did break. This is okay. There was a major version bump. +Of course compiled modules would need to be redone after a change like +that. I think the way that Python handles Unicode in version 3 is ideal +and should be an example for other languages. + +Virtualenv and pip is not as bad as using bundler and gem for Ruby. +Virtualenv very clearly makes changes to your environment variables that +are easy to compare and inspect. This is in contrast to the ruby tools +that encourage global modifications of your shell and supercede the +packaged versions of the language interpreter. + +The sad part is that I see [this pattern of senseless locking of +versions continuing +elsewhere](https://github.com/tools/godep) instead of proper +maintenance of libraries and projects. + +### Insanity + +To make matters worse, people suggest you actually embed all the source +code for every dependency inside the repository. Meaning your commit +graphs and code line counts are skewed based on the contents of your +upstream packages instead of just the code you wrote. Admittedly, +locking dependencies like this does mean that fantastic language level +tools such as [go +get](https://golang.org/cmd/go/#hdr-Download_and_install_packages_and_dependencies) +work again, but overall it is just not worth the pain +of having to manually merge in patches from upstream (but if you do +think it is worth the pain contact me, I'm open for contract work) +making sure to change the file paths to match your changes. + +### The Solution + +I believe the solution to all this and something that needs to be a +wider community effort for users of all programming languages is the use +of a technique called [semantic +versioning](http://semver.org/). In +some lanaguages like Go where the [import paths are based on repository +paths](https://golang.org/doc/code.html#PackagePaths), this may mean that +a new major version has a different repository. This is okay. Backward +compatability is good. After you make a stable (1.0 or whathaveyou) +release, nothing should be ever taken away or changed in the public API. +If there needs to be a change in how something in the public API works, +you must keep backwards compatabilty. As soon as you take away or modify +something in the public API, you have just made a significant enough +change worthy of a major release. + +We need to make semver a de-facto standard in the community instead of +freezing things and making security patches hard to distribute. + +Also, use the standard library more. It's there for a reason. It doesn't +change much so the maintainers are assumed to be sane if you trust the +stability of the language. + +This is just my \$0.02. |