Makefile: don't run git checkout on behalf of the user

[dmerejkowsky]

I know this is safe because we check the repo is clean, but don't like code I run on a repo to change the git state
of another repo nearby.

Maybe that's just me, though ...

[dmerejkowsky]

On the other hand, this adds more Make code to maintain, which is not ideal ...

[JulienPalard]

I get that it can be surprising or even worrying to have a repo being changed without much warning, like "have I been pwned? Why is my cpython repo in detached state? It was not 10mn ago !?".

My first impression was "it's OK, translators may not be cpython contributors, they won't even notice." but I may be wrong, translators may spot upstream issues, and fix them themself (it happened quite a few times actually).

@deronnax, @Seluj78, @christopheNan any feelings on this?

The other way around is cloning a whole new cpython hidden in a gitignored directory say .cpython, so we can play with it without user interaction. A bit like we did when we used git worktrees.

[christopheNan]

Considering the SolarWinds attack where the build system was compromised, I consider that nowadays it is not a good practice to modify a repo outside of the project.
I would then recommend to host the cpython clone in a subtree of python-docs-fr. Maybe a cherrypicking of cpython is sufficient?

[JulienPalard]

Maybe a cherrypicking of cpython is sufficient?

what do you mean?

[christopheNan]

Maybe a cherrypicking of cpython is sufficient?

what do you mean?

Sorry, I was confused with a concept of Centralised VCS (where you can extract a light working copy by cherrypicking on the repository).
My opinion is that it is worth creating a cpython clone for the python-docs-fr tree.

[JulienPalard]

My opinion is that it is worth creating a cpython clone for the python-docs-fr tree.

I tried to avoid it because it's time consuming, I mean git clean -dfqx and boom it's gone, but hey, it's the initial goal of a git clean -dfqx so why not.

That's also why lots of previous Makefiles attempts were trying to do shallow clones, but as we need a specific commit and there's no git clone --since <revision> it was always ugly hacks.

So I'd go for a full clone. Or maybe --shallow-since 2020-01-01 something like that, a hardcoded date that we know contain what we need... ?

Should we call it .cpython?

[christopheNan]

or --single-branch?

[JulienPalard]

or --single-branch?

Pas sûr que ça élague beaucoup, la branche 3.9 à ses racines "jusqu'au début". Bon "pas sûr" n'étant pas un argument très solide j'ai testé :

490M	cpython-clone/
417M	cpython-3.9/

pas top.

[JulienPalard]

@dmerejkowsky What about the discussed idea of having a specific cpython clone inside the python-docs-fr hierarchy, hidden using the .gitignore?

• Drawbacks the current method was trying to avoid: duplicate a clone for those already having one (no much people I bet).
• This drawback can be attenuated using --shallow-since with a date like 2020-01-01.
• benefit: a git clean -dfqx actually clean so one can really start from scratch when something goes bad.

[dmerejkowsky]

.... or we could fetch https://github.com/python/cpython/archive/{sha}.zip

[JulienPalard]

.... or we could fetch https://github.com/python/cpython/archive/{sha}.zip

Ça fait 26MB, c'est pas idiot. Bon ça nécessite un wget et un unzip qui fonctionnent, on peut préjuger ça sur MacOS ?

[dmerejkowsky]

Ça fait 26MB, c'est pas idiot. Bon ça nécessite un wget et un unzip qui fonctionnent, on peut préjuger ça sur MacOS ?

Non on peut pas. Mais on a Python, urllib et zipfile :)

[JulienPalard]

Non on peut pas. Mais on a Python, urllib et zipfile :)

Alors oui. Mais ce serait super si le Makefile restait simple et lisible, donc bof ?

Je comprend que sur certains gros projets en C le Makefile "grossisse" un peu, pour de la doc, bof.

[dmerejkowsky]

Après ça se tente avec curl et en priant pour que le tar macOS casse pas trop les pieds. Mais j'ai pas de matériel Apple disponible pour tester.

[JulienPalard]

git clone https://github.com/python/cpython --shallow-since 2020-01-01
git -C cpython checkout eec8e61992fb654d4cf58de4d727c18622b8303e
rm -fr cpython/.git

prend exactement la même place sur le disque que le zip (normal : on obtient exactement les mêmes fichiers, dans la même hierarchie). Ça nécessite probablement plus de bande passante et de CPU :

• wget + unzip : 7s
• par git : 14s

Ah sinon au lieu de hardcoder une date on pourrait fetch le bon commit :

time (mkdir foo; cd foo; git init; git remote add origin https://github.com/python/cpython; git fetch origin eec8e61992fb654d4cf58de4d727c18622b8303e; git reset --hard eec8e61992fb654d4cf58de4d727c18622b8303e; rm -fr .git)

mais ... ça prend 54s ☹

Donc peut être faire le wget comme tu dis, mais en .tar.gz pas en .zip, j'ai plus confiance dans le fait que tout le monde a tar d'installé que unzip.

[JulienPalard]

Attention j'ai rebasé sur la 3.10.

[JulienPalard]

On utilise notre propre clone de cpython, caché dans .venv/, comme ça on est pas embêté par l'utilisateur qui touche a son cpython/, ni on embête l'utilisateur a toucher a son cpython/.