Skip to content

[DEPR] Devstack #247

Description

@kdmccormick

Timeline

  • Proposed: 2022-03-01

  • Updated Acceptance Date: 2024-03-12

  • Archived: 2024-03-19 (pre-Redwood)

    • Anyone wishing to continue to use devstack will have to make their own forks in order to make changes or keep it updated/maintained.
    • 2U will likely maintain its own fork, but there will be a number of 2U-specific changes and will not be open to outside development. It is possible that this fork will be made private.
  • Removal of devstack support from application repositories:

    • Original deadline: 2024-10-10
    • New deadline: 30 days after lms/envs/development.py and cms/envs/development.py are merged into edx-platform master.
      • Estimate: development.py will merge 2024-10-20, old devstack.py will be removed on 2024-11-20.

Replacement

The replacement for Devstack is Tutor. Tutor is "the Docker-based Open edX distribution designed for peace of mind". It is both a deployment tool and a development environment. Tutor has been the only community-support production deployment method since Maple, although the Open edX docs and website do not officially recommend it as a development environment yet.

Rationale

Devstack has several issues:

  • It is not pluggable. This leads to two issues:
    • It is large. Out of the box, it comes with a dozen IDAs and another dozen MFEs; new Open edX developers generally needs two IDAs (LMS and CMS) a few MFEs. In the past couple years, improvements have been made such that only a subset of IDAs are run by default. Still, cloning and provisioning are bound by the full weight of all services.
    • Some IDAs like enterprise-catalog, in order to avoid being part of the large Devstack core, make a plugin-esque docker-compose.yml file that hooks into Devstack. This complicates the developer experience on those IDAs and also makes it harder to modify Devstack itself (because of the implicit plugin interface that such IDAs are using).
  • Its images are slow to download. The Docker images used by devstack are built by running Ansible in a Dockerfile instead of using native Dockerfile commands. This means that the images are large and make poor use of the Docker layer cache, and thus take a long time for Devstack users to download.
  • Many users find it difficult to diagnose and fix Devstack errors. Often, they find it more time/effort-efficient to destroy their volumes and provision from scratch instead of fixing the immediate issue.
  • Provisioning takes a long time. This exacerbates the previous issue.
  • There are several "upkeep" steps that must be independently done on a regular basis in order to keep Devstack working reliably. These include git-pulling devstack, git-pulling multiple app repos, pulling Docker images, and running migrations.
  • Setting up a service in Devstack is completely tangential to setting it up for production deployment. Once it is set up, it takes attention and effort to keep the setup consistent between Devstack and production.

Tutor shows promise in many of these areas:

  • It features a documented plugin API. Many IDAs (forums, discovery, credentials, et al) are implemented as plugins to Tutor. The core of Tutor includes just LMS and CMS, and there is even work being discussed to factor those out into plugins.
  • Tutor's Dockerfiles are written using native Dockerfile syntax and are intentionally designed to be as small and cache-friendly as reasonably possible.
  • Tutor has actively-maintained documentation including a Troubleshooting section. Beyond that, Tutor has a dedicated forum community where maintainers and users help each other. Tutor also makes a point to have a consistent, documented, and debuggable CLI. For example, the tutor dev segment of the CLI is implemented entirely on top of docker-compose and transparently prints out the underlying docker-compose commands that it runs to aid in user debugging, using terminal colors to call out commands vs tutor logging vs actual output.
  • Once Tutor is installed, provisioning involves one command (tutor local quickstart) and runs fairly quickly. Provisioning can be safely re-run without destroying existing data.
  • Because Tutor uses the code in the Docker images by default, the user only needs to regularly git-pull the Tutor repo and the repos that they host-mount in, if any.
  • Since Tutor is also the community-supported deployment method, one can support development and and production usage of an IDA all in a single plugin. Furthermore, using Tutor for both dev and prod allows operators to keep their dev and prod environments very similar with less effort.

For areas where Tutor could be improved, there is an ongoing Tutor Adoption Initiative which aims to support the transition from Devstack to Tutor through a mix of education, plugin development, and core Tutor improvements. Within the initiative roadmap, the "Dev Quality-of-Life" epic contains stories that should ideally close any existing gaps between Tutor and Devstack.

Migration

Many developers can start using Tutor now, and many already are). Particularly, developers whose projects are encompassed by the LMS, CMS, MFEs, and IDAs covered by existing official plugins can generally transition their workflows over to Tutor. On the other hand, for teams that run IDAs with less community adoption (notably, teams at 2U), several new Tutor plugins will be needed in order for Tutor to fully replace Devstack.

There are no plans to help users migrate data from a Devstack instance to a Tutor instance. We recommend that folks start fresh when switching from Devstack to Tutor.

Deprecation & Removal

Deprecation steps

Removal steps

  • Archive https://github.com/openedx/devstack
  • Update OEP-10, which still mentions "devstack" as a supported distro
  • Update dev setup in README openedx-platform#34430
  • Remove env.devstack files from frontend apps
  • Scour our docs for other references to devstack; update them to Tutor.
  • Delete the various devstack.py and development.py settings files unless otherwise depended upon.
  • Delete the various docker-compose.yml files and Makefile targets that some IDAs (eg enterprise-catalog) use in order to integrate with Devstack.
  • Delete the docker/ role (incl. Dockerfiles) from github.com/openedx/configuration.
  • Follow up: Decide whether or not to keep the Dockerfiles that exist in various repos, which aren't used in Tutor.
  • Update edx-cookiecutters

PRs related to replacing devstack.py with a more general working development.py

Activity

  1. added
    deprProposal for deprecation & removal per OEP-21
    on Feb 28, 2022
  2. changed the title [-][DEPR]: Devstack[/-] [+][DEPR] Devstack[/+] on Feb 28, 2022
  3. kdmccormick commented on Feb 28, 2022

    @kdmccormick
    MemberAuthor
  4. dianakhuang commented on Jul 13, 2023

    @dianakhuang

    Since this is primarily a 2U project right now, we are planning on announcing this and gathering feedback.

    2U is still planning on using this for the near to medium term until we're confident we can migrate successfully onto Tutor, but we can move it out of the openedx GitHub org to minimize confusion.

  5. kdmccormick commented on Jul 13, 2023

    @kdmccormick
    MemberAuthor

    In general, sounds great!

    Only thing is... MFE development currently sucks in Tutor >.< So nearly all frontend devs still use Devstack when they're trying to change MFEs.

    Brian Smith is actively working on rectifying this, with promising results. Would you be willing to delay the deprecation further until Tutor has MFE dev sorted out?

  6. dianakhuang commented on Jul 13, 2023

    @dianakhuang

    Yup! I think we're still trying to understand the requirements needed here. I'll push out the acceptance date for another few months.

  7. dianakhuang commented on Aug 28, 2023

    @dianakhuang
  8. kdmccormick commented on Aug 31, 2023

    @kdmccormick
    MemberAuthor

    @dianakhuang So, I know I had originally been a proponent of deprecating both configuration and devstack "into the edx org", under the understanding that if only 2U is using some parts of the project, then it's easier for both sides if we transfer those parts over to 2U. Since then, I've talked about this with my team at Axim, and come to understand that the situation is more nuanced than that.

    The code in the openedx org is built from a mix of Axim-owned contributions and community-contributor-owned contributions (licensed to Axim under the CLA). We hold the code in this org for the good of all current & former Open edX contributors and users, and as the project's stewards we have a mandate to maintain ownership and ensure availability of all its repositories, even unsupported repositories.

    Our procedure of archiving repos by transferring to the openedx-unsupported org lets us mark a repo as "unsupported" while still guaranteeing the repo's availability and keeping its ownership clear. So, if/when devstack and configuration are deprecated, we'll need to move them there.

  9. dianakhuang commented on Aug 31, 2023

    @dianakhuang

    @kdmccormick that sounds reasonable! 2U/edX.org should be able to then clone the repositories for our own use case after they are put into unsupported, then?

    Obviously, this is our first stab and understanding how this process is going to work going forward.

  10. kdmccormick commented on Aug 31, 2023

    @kdmccormick
    MemberAuthor

    Yes, you could fork from openedx-supported after that full deprecation & archival process.

    Even in that case, though, I'd be curious what we could do to help keep 2U involved with community tooling.

    Would you be forking devstack & configuration as a temporary stop-gap until you're able to transition off of them, or with the intent of using them indefinitely?

    • If you're using them as a stop-gap, then I bet some other community members are also using them as a stop-gap. It could be worth keeping these repos un-archived until the replacement options (Tutor or otherwise) are viable to all or almost all community members.
    • If you plan to use them indefinitely, then I bet some some other community members would see value in using them indefinitely, too, which I would see as an argument for keeping the repos un-archived.

    Obviously, this is our first stab and understanding how this process is going to work going forward.

    Same here :) learning as we go!

  11. dianakhuang commented on Jan 11, 2024

    @dianakhuang

    We are going to block acceptance because devstack is still used heavily by people who need to do development on MFEs outside of 2U.

    @arbrandes @kdmccormick could you add more context on this?

  12. 43 remaining items

  13. robrap commented on Jan 29, 2025

    @robrap

    [inform]

    • The cookie-cutter ultimately needs to be updated to match whatever final pattern you want around docker for new IDAs.
    • Related, when creating the new codejail-service, we simply removed all the docker/devstack-related code. It may one day want to get updated with whatever pattern is created.
  14. kdmccormick commented on Jan 29, 2025

    @kdmccormick
    MemberAuthor

    @robrap

    • Good point, I added a task to the issue description
    • Thank you for doing that!
  15. kdmccormick commented on Feb 14, 2025

    @kdmccormick
    MemberAuthor

    Update: While devstack itself is deprecated, there are still many tendrils to clean up in the openedx org , such as:

    • outdated docs.
    • devstack django settings files , whose removal is blocked by Simplify edx-platform settings openedx-platform#36215
    • devstack-specific env.development files in the frontend-apps. Their removal is not imminent, but it also not blocked by any upstream concerns. I encourage any users of devstack forks to consider how they could either make those files tooling-agnostic (perhaps using local.openedx.io) or how they could operate devstack without those files being in the upstream repos.
  16. robrap commented on May 16, 2025

    @robrap

    @kdmccormick: I'm wondering if we could do away with the "blocked" status, and these tickets are helping me get more clear.

    I think this ticket could move to Transition Unblocked and when updating to the new DEPR template, could get a task list for unblocking removal, and a task list for completing the removal. The first list would be required to get to Breaking Changes Unblocked. The Review After Date could simply be used as some future check-in date. Thoughts?

  17. moved this from Blocked to Transition Unblocked in DEPR: Deprecation & Removalon May 16, 2025
  18. kdmccormick commented on May 16, 2025

    @kdmccormick
    MemberAuthor

    Yes, I think that sounds reasonable! I updated the ticket status. I can add that info to the the tasklist in the future (I plan to update all my tickets in one go when I have a couple spare hours).

  19. dianakhuang commented on Jun 26, 2025

    @dianakhuang
  20. moved this from Transition Unblocked to Breaking Changes Unblocked in DEPR: Deprecation & Removalon Jan 8, 2026
  21. moved this from Breaking Changes Unblocked to Transition Unblocked in DEPR: Deprecation & Removalon Jan 8, 2026
  22. moved this from Transition Unblocked to Plan Accepted in DEPR: Deprecation & Removalon Jan 8, 2026
  23. dianakhuang commented on Jan 8, 2026

    @dianakhuang

    This is waiting for new development environment settings files to be created. Blocked on the edx-platform settings refactoring.

  24. added theissue type on Jan 28, 2026
  25. dianakhuang commented on Jun 25, 2026

    @dianakhuang

    We need a new base development settings file before this can be completed. The creation of this settings file is no longer blocked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

deprProposal for deprecation & removal per OEP-21

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions