XBND-1187: refactor!: async multi-threaded logs + major prettify upgrade
[x] I tested/built with start-docker.ps1
or docs/make.bat
and confirm the expected outcome.
High-Level Proposal*
This is essentially an overhaul to apply what I learned about Python as SphinxManager progressed, offering a major pass on code readability/quality followed by a log revamp created specifically for async, multi-threaded environments.
Release Notes*
-
Revamped
sphinx_repo_manager.py
+git_helper.py
- Broke down large funcs
- More-intuitive, scalable, readable code
- Removed unnecessary comments
-
Nuked the existing logging system and replaced with:
- an async, multi-threaded friendly variant using progress bars and animations that are perpetually at the bottom of the screen
- For git clones, we explicitly track the process by parsing the
--progress
% returned. CC @rdumouchelle (big repo progress tracking)
- For git clones, we explicitly track the process by parsing the
- More intuitive, less details, less spammy on success
- Dumps error info on fail
- an async, multi-threaded friendly variant using progress bars and animations that are perpetually at the bottom of the screen
-
Instead of globally ignoring .idea (JetBrains IDE), surgically selected some files that collaborators can use to globally normalize/prettify the same way.
Media
Tests
- Tested with
make html
with 5 workers - Tested cloning with tags
- Tested cloning without tags
- Tested updating existing repos with tags
- Tested updating existing repos without tags
- Full cloning of 31 repos takes ~56s
Regression Warning
-
⚠ ️ We used to have regeneration for when a dir within_repos-available
andcontent
. Now if you delete an individual dir, it's currently not regenerated.- Since this is non-essential, this should be a separate task // TODO: Add to Jira
- To workaround, simply delete both
_repos-available
andcontent
for a clean [re]clone.
Edited by Dylan Hunt