Let’s say you built an amazingly wonderful hand-crafted artisanal convolutional neural network that works beautifully on your hard-drive based dataset. You are now ready to take this masterpiece to the cloud to work on a much larger dataset on beefier machines - you are not looking forward to it.
- “How do I ship this patchwork conda (or venv) where I’ve installed everything the training/inference code needed AS WELL AS everything else I thought I needed along the way?”
- “I don’t want to waste cloud compute money on things I’m not sure will work on the first try!!”
- Basically, “I have already done the work and I’m not interested in the yak shaving portion of the job!”
No need to fear, dear reader, this article is designed to help you move your glorious work to the cloud (and beyond) by leveraging your local environment as if it were the cloud itself.
Making software is hard. Every once in a while we find some code that really helps the process. It makes our own code intelligible: almost fun! Sometimes the magical code is of our own making but often we are using someone else’s code. In either case the next step should be to share the joy.
My preferred mechanism to share the joy is through videos. Videos are a powerful way of humanizing the types of problems software solves. With video, you can get in the thick of it code-wise while still adding a literal face. You can see the excitement, the technology, and most importantly: the code.