Improved and expanded runtime documentation #303
Conversation
…time I'd like to make this distinction even clearer in the future, but it'll have to wait for another doc push. For now, at least provide a pointer to anyone coming to this page wanting to update the runtime.
Trying to provide a more useful resource to point people to than a smattering of scattered doc snippets. There's still some places this could be better cross-referenced (e.g. in --help output) and more that could be said about runtimes in general and each runtime specifically… but that can all come in due time. I needed to find a stopping point for this work _somewhere_. A big gain here is that now there's clear places to attach additional documentation in the future. Like the recent switch from dynamically- to statically-generated rST for command documentation, it may also sooner-than-later make sense to ditch the "automodule" directive and do something static instead. Resolves <#288>.
Previously undocumented!
Since we immediately convert the Docker/OCI image to SIF, the Docker/OCI image layer caches end up being duplicative. Not keeping them cached means the cache can't be used to avoid downloading layers that haven't changed in subsequent updates, but I think when choosing between minimizing transfer or storage for our users, we should minimize storage. If we want to have our cake and eat it to (i.e. minimize both), we might be able to change tack to newer Singularity features where (I think) OCI images can be used directly without first converting to SIF. Then, like with our Docker runtime, the image we use and layer caches are one and the same. That's a larger change for a future time, though.
|
Pushed another couple smaller commits addressing #225. |
…ary with ours More internally consistent now that we have the glossary term, which in turn links to our overview page about them for more details.
There was a problem hiding this comment.
This will be super helpful for users to learn more about the different runtimes 🌟
This is definitely out-of-scope of this PR, but as I was reading the new docs I kept wondering if we should list the non-Nextstrain tools that are available in the runtimes. It's definitely discoverable if people go digging in the Dockerfile or Conda recipe.yaml but might be helpful to list the commonly used tools such as tsv-utils, csvtk, and seqkit.
doc/runtimes/index.rst
Outdated
|
|
||
| Switching runtimes or updating the version of a runtime may result in different | ||
| versions of Nextstrain components like Augur and Auspice as well as other | ||
| programs, and thus different behaviour. |
There was a problem hiding this comment.
My immediate questions after reading this part:
- How do I know which versions are available in my current runtime?
- How can I find a version of a runtime that includes the specific version of a Nextstrain program that I want?
For [1], I think it'd be helpful to move or recap the sentence about nextstrain version --verbose here.
Not sure others may raise [2] but wanted to put it out there.
There was a problem hiding this comment.
I found myself asking [2] recently when Augur 22.2.0 had a bug. It took me some digging in the Docker image's build logs to figure out that nextstrain/base:build-20230720T001758Z was the last runtime version with an earlier version of Augur.
I don't think there's an easy solution here.
One solution would be to maintain an up-to-date resource describing the versions of Nextstrain programs in each runtime version. For example:
| runtime | version | Augur | Auspice | Nextstrain CLI |
|---|---|---|---|---|
| conda | nextstrain-base 20230815T151610Z | 22.3.0 | 2.46.0 | 7.1.0 |
| conda | nextstrain-base 20230805T043657Z | 22.2.0 | 2.46.0 | 7.1.0 |
| docker | nextstrain/base:build-20230814T140428Z | 22.3.0 | 2.46.0 | 7.1.0 |
| docker | nextstrain/base:build-20230805T043648Z | 22.2.0 | 2.46.0 | 7.1.0 |
| … | … | … | … | … |
This would be straightforward to use, but probably not straightforward to maintain.
There was a problem hiding this comment.
Pushed 8b90a0b to address 1. Addressing 2 seems useful, I agree, but I think to provide a useful answer besides "binary search it yourself" we'll need to produce static reports for each runtime version of the included software (ours and others). This would also address your broader comment above.
(Update: I wrote this without seeing Victor's comment due to a stale tab! I do think such a report, like his example or otherwise, would be sustainable to maintain automatically.)
doc/runtimes/index.rst
Outdated
|
|
||
| Switching runtimes or updating the version of a runtime may result in different | ||
| versions of Nextstrain components like Augur and Auspice as well as other | ||
| programs, and thus different behaviour. |
There was a problem hiding this comment.
I found myself asking [2] recently when Augur 22.2.0 had a bug. It took me some digging in the Docker image's build logs to figure out that nextstrain/base:build-20230720T001758Z was the last runtime version with an earlier version of Augur.
I don't think there's an easy solution here.
One solution would be to maintain an up-to-date resource describing the versions of Nextstrain programs in each runtime version. For example:
| runtime | version | Augur | Auspice | Nextstrain CLI |
|---|---|---|---|---|
| conda | nextstrain-base 20230815T151610Z | 22.3.0 | 2.46.0 | 7.1.0 |
| conda | nextstrain-base 20230805T043657Z | 22.2.0 | 2.46.0 | 7.1.0 |
| docker | nextstrain/base:build-20230814T140428Z | 22.3.0 | 2.46.0 | 7.1.0 |
| docker | nextstrain/base:build-20230805T043648Z | 22.2.0 | 2.46.0 | 7.1.0 |
| … | … | … | … | … |
This would be straightforward to use, but probably not straightforward to maintain.
…ng/updating runtimes A good suggestion by @joverlee521 in review.
à la how we do it for the docker-base repo and the Docker runtime. A good suggestion by @joverlee521 in review.
…f our ambient runtime The general installation page contains a good example of ambient setup. A good suggestion by @victorlin in review.
For this, I think I will limit work for now to adding a FAQ entry in the umbrella docs and cross-linking into this CLI reference documentation. |
…the new runtimes documentation This hopefully clarifies a term we use without really explaining it, and at least provides a pointer to more details for those who want to know more. Related-to: <nextstrain/cli#303>
…the new runtimes documentation This hopefully clarifies a term we use without really explaining it, and at least provides a pointer to more details for those who want to know more. Related-to: <nextstrain/cli#303>
See commit messages.
Resolves #288.
Future PRs?
doc/aws-batch.md) and the AWS Batch runtime page (Integrate AWS Batch set up and usage doc with runtime doc #304)Testing