Skip to content

XBND-1187: refactor!: async multi-threaded logs + major prettify upgrade

[x] I tested/built with start-docker.ps1 or docs/make.bat and confirm the expected outcome.

High-Level Proposal*

This is essentially an overhaul to apply what I learned about Python as SphinxManager progressed, offering a major pass on code readability/quality followed by a log revamp created specifically for async, multi-threaded environments.

Release Notes*

  1. Revamped sphinx_repo_manager.py + git_helper.py

    • Broke down large funcs
    • More-intuitive, scalable, readable code
    • Removed unnecessary comments
  2. Nuked the existing logging system and replaced with:

    • an async, multi-threaded friendly variant using progress bars and animations that are perpetually at the bottom of the screen
      • For git clones, we explicitly track the process by parsing the --progress % returned. CC @rdumouchelle (big repo progress tracking)
    • More intuitive, less details, less spammy on success
    • Dumps error info on fail
  3. Instead of globally ignoring .idea (JetBrains IDE), surgically selected some files that collaborators can use to globally normalize/prettify the same way.

Media

clone-example

clone-example-done

Tests

  • Tested with make html with 5 workers
  • Tested cloning with tags
  • Tested cloning without tags
  • Tested updating existing repos with tags
  • Tested updating existing repos without tags
  • Full cloning of 31 repos takes ~56s

Regression Warning

  • ️ We used to have regeneration for when a dir within _repos-available and content. Now if you delete an individual dir, it's currently not regenerated.
    • Since this is non-essential, this should be a separate task // TODO: Add to Jira
    • To workaround, simply delete both _repos-available and content for a clean [re]clone.
Edited by Dylan Hunt

Merge request reports