starseeker (Sep 06 2020 at 15:46): starseeker (Sep 06 2020 at 15:46):I think I got the github web repo set up to use LFS (anyone with existing forks/checkouts will probably have to rebase).
@Sean I don't think I can see the org capacity settings for LFS - if I'm reading the docs right it's actually a 1GB size limit, so I'm not sure how useful that's really going to be...
Sean (Sep 06 2020 at 19:19):lets see if it's actually a limit
Sumagna Das (Sep 06 2020 at 19:23):yea i also read that 1 gb is the limit for lfs for orgs if i remember correctly
Sean (Sep 06 2020 at 19:24):I'm just wondering if it's a hard limit. They used to have documented that there was a limit on repos to, but then that turned out to just be documentation, not an actual limit imposed.
Sean (Sep 06 2020 at 19:25):it was a soft limit to get communities to at least try to not use too much space, not a limit on orgs that genuinely need to go over
Sean (Sep 06 2020 at 19:25):could be a hard limit. easy enough to find out.
Sean (Sep 06 2020 at 19:26):Here's 1GB of docs... https://brlcad.org/BRL-CAD_V_and_V_Documentation.7z
Sumagna Das (Sep 06 2020 at 19:27):is this seriously a 1gb file?
Sean (Sep 06 2020 at 19:30):1024172845 bytes, 977M
Sumagna Das (Sep 06 2020 at 19:30):ooh
Sean (Sep 06 2020 at 19:37):nearly all our docs related to verification and validation, official report published, user docs, administrative docs, etc.
Sean (Sep 07 2020 at 05:47):@starseeker What's up with all the web repo closures? Some of the ones outstanding needed work while others were conflicting efforts (some that broke others). Did you clean them up?
starseeker (Sep 07 2020 at 15:44):I didn't deliberately close them - it looks like I may have inadvertently done that while enabling the LFS bits.
starseeker (Sep 07 2020 at 15:47):Crud.
starseeker (Sep 07 2020 at 16:34):@Sean I tried to restore the repository to the point before I made changes, but doesn't seem to have allowed me to reopen the pull requests.
starseeker (Sep 07 2020 at 16:38):They might have to be resubmitted, but the repo history should now again match what's being used in the forks
starseeker (Sep 07 2020 at 16:43):We'll have to decide what we're going to do about that at some point - it looks like to "properly" migrate the repo to LFS the change is disruptive. Maybe the better thing to do is just make a separate repo for the timeline and other doc heavy content and then have the web repo treat it as a submodule?
starseeker (Sep 07 2020 at 16:46):I guess the thing to do is post an explanatory note on the closed pull requests so people know to resubmit them?
Sean (Sep 07 2020 at 21:21):I doubt people will, super confusing, but maybe. Seems odd that it can't be undone -- this didn't work? https://help.github.jp/enterprise/2.11/user/articles/deleting-and-restoring-branches-in-a-pull-request/
Sean (Sep 07 2020 at 21:21):in theory the branch and the commits should still be there
starseeker (Sep 07 2020 at 21:22):I doubt it because of the way I had to restore.
Sean (Sep 07 2020 at 21:23):what do you mean? what'd you do? o.O
Sean (Sep 07 2020 at 21:24):yeah, it's saying "this branch has no history in common with BRL-CAD:master" .. so it won't allow them to be reopened
starseeker (Sep 07 2020 at 21:25):When I did the LFS migration I rewrote part of the Git history - when you pointed out that closed the patches I put back an older forked copy without those changes, but Github doesn't recognize that the old copy is back
Sean (Sep 07 2020 at 21:26):oh man
starseeker (Sep 07 2020 at 21:26):The SHA1s should match again, so the forks will "see" the same repo they forked from, but Github doesn't know that.
starseeker (Sep 07 2020 at 21:29):My apologies - I should have foreseen that as a possible consequence, but it didn't occur to me
Sean (Sep 07 2020 at 21:29):according to the closed pull requests, it does know and sees them as yet another forced push that doesn't match the original
starseeker (Sep 07 2020 at 21:30):No, those are both from yesterday - the revert push was earlier today
Sean (Sep 07 2020 at 21:31):ah, hrm
starseeker (Sep 07 2020 at 21:31):If you check out one of the forks, and clone the current repo, and look at the SHA1 in the logs, you should see that they match now.
starseeker (Sep 07 2020 at 21:33):That wiped out the timeline commits of course, but that won't matter - the key point is the history that the forks are using should be back. It's just not a situation the github pull system was designed for.
Sean (Sep 07 2020 at 21:34): starseeker (Sep 07 2020 at 21:35):Can I do that though, or does the submitter need to do it?
Sean (Sep 07 2020 at 21:36):I wouldn't think they should need to. The info should be accessible.
Sean (Sep 07 2020 at 21:39):looks like you might have to do it locally, commit to a new branch, push that new branch, and then do a PR from the new branch (i.e., recreate it on their behalf).. which is probably as much work as simply cherry picking all their commits and merging properly
starseeker (Sep 07 2020 at 21:40):Except I thought the pull requests weren't ready to be merged?
Sean (Sep 07 2020 at 21:40):man sh!t happens, but that really sucks... there's at least weeks of work in those pull requests.
starseeker (Sep 07 2020 at 21:41):The work isn't lost
starseeker (Sep 07 2020 at 21:42):That's an aspect of Git I'm not familiar with, but I can try to grab their patches and submit pull requests.
Sean (Sep 07 2020 at 21:42):they were all in different states. some just simply hadn't been reviewed or tested. some had minor conflicts because another pull request made theirs non-automatic. others had undesirable changes mixed in with super desirable changes (like the big front-page cleanup -- it was awesome except it changed the menu not for the better imho)
starseeker (Sep 07 2020 at 21:42):I'm just leery of screwing something up even worse.
Sean (Sep 07 2020 at 21:43):probably safe if you don't use a force flag, no history rewriting... ;)
Sean (Sep 07 2020 at 21:44):I still don't even fully understand what happened.. this is not something I've seen before.
starseeker (Sep 07 2020 at 21:44):Well, let me try one and see - so I do a clone, use gh pr to get the patch, apply it, and then push? Or do I fork to my own account, apply the patch, and submit from my own account?
starseeker (Sep 07 2020 at 21:44):It's my screw-up, pure and simple.
Sean (Sep 07 2020 at 21:44):maybe wait and see if github catches up and lets them be reopened automatically
Sean (Sep 07 2020 at 21:45):could be a latent update process that takes time for it to notice and update all caches
starseeker (Sep 07 2020 at 21:45):I tried to make LFS the default for our PDF files, and before doing that I scrubbed four spam PDFs out of the history so they wouldn't get fossilized in LFS. Then I did a migrate so the big existing PDFs would also be on LFS. Both of those ended up rewriting the history, which broke the SHA1s, which broke the pull requests.
Sean (Sep 07 2020 at 21:46):starseeker said:
Well, let me try one and see - so I do a clone, use gh pr to get the patch, apply it, and then push? Or do I fork to my own account, apply the patch, and submit from my own account?
yeah, that I'm not sure about
starseeker (Sep 07 2020 at 21:47):I didn't save quite an old enough backup, so I had to restore the history from one of the forks - that worked, but may not have saved any internal pull request info github stashed in the repo (if that's what it needs to "restore" the pull requests.)
Sean (Sep 07 2020 at 21:48):https://github.com/letsencrypt/website/issues/639
Sean (Sep 07 2020 at 21:48):seems to indicate it updates slowly, might be reopenable on a case-by-case given enough time
starseeker (Sep 07 2020 at 21:49):OK, so give it a week or so looks like?
Sean (Sep 07 2020 at 21:49):3 days should be plenty
Sean (Sep 07 2020 at 21:51):if it's not by then, I suggest we simply work on cherry-picking all the commits in those half dozen prs to close them out. shouldn't take more than a day I'd hope to review and cull the undesirable bits and merge the rest.
starseeker (Sep 07 2020 at 21:54):Sounds like a plan, although I'm not skilled enough at web stuff to do those merges with any assurance of a working output...
starseeker (Sep 07 2020 at 21:54):Or do you mean cherry pick them as branches on the main repo and merge from there?
Sean (Sep 07 2020 at 21:56):I mean cherry pick and merge/commit the work (i.e., lets just process the pull requests) ... or alternatively create new PRs with the commits, so it doesn't get lose out-of-sight-out-of-mind.
Sean (Sep 07 2020 at 21:57):it's pretty simple to clone out the repo and test from some ~/public_html subdirs, or even clone locally and point firefox. I don't think we have any remaining dynamic bits.
starseeker (Sep 07 2020 at 21:57):Well, the cute trick doesn't work:
gh pr checkout 37
git push origin HEAD:refs/pull/37/head
! [remote rejected] HEAD -> refs/pull/37/head (deny updating a hidden ref)
error: failed to push some refs to 'git@github.com:BRL-CAD/web.git'
starseeker (Sep 07 2020 at 21:58):Can't restore the pr that way
Sean (Sep 07 2020 at 21:59):what was this all about? git pull origin master --allow-unrelated-histories
Sean (Sep 07 2020 at 21:59):might that tell origin to allow them?
starseeker (Sep 07 2020 at 21:59):That will pull the upstream into a locally altered copy of the repository, even though the local SHA1s don't have anything to do with the upstream
starseeker (Sep 07 2020 at 22:00):"push" would try to send something back to origin, but I did that accidentally once working with my local cad conversion tests - it really made a mess
starseeker (Sep 07 2020 at 22:17):@Sean Will this work? https://github.com/BRL-CAD/web/pull/50
starseeker (Sep 07 2020 at 22:19):Doesn't re-create the comment history in the pull request, but if We link to the closed PR in the comment everything's still tied together.
starseeker (Sep 07 2020 at 22:33):If that's acceptable that's not too hard to do, and should restore all of them - it'll show them coming from my account, but it looks like the commits are all correctly attributed to their actual author.
Main problem would be if there is more work done by the original authors in their downstream forks - in that case they'd still need to make a new pull request.
starseeker (Sep 10 2020 at 19:29):@Sean I think I've got everything resubmitted as pull requests
Sean (Sep 11 2020 at 03:21):okay, thank for the reparo .. I really need to spend a day merging outstanding requests soon. :/
starseeker (Sep 11 2020 at 12:27):/me nods. At some point we also need to decide what to do about LFS and scrubbing out spam files from the history (this time with an awareness of the consequences). Even with the pull requests closed cleaning that up will still inevitably diverge the repo from external forks regardless - people will need to "re-fork" and rebase. We might be able to turn on LFS just for files going forward, but when I tried that each clone operation printed out a list of files that matched the LFS filters that hadn't been migrated (harmless but rather annoying). Even if we did that though, it still would leave the spam files in the existing repo history.
starseeker (Sep 11 2020 at 12:29):Also, do we want to consider using submodules for specific pieces of the website (like the timeline) so we can alter portions of the site more radically without disturbing the full history?
Last updated: Oct 09 2024 at 00:44 UTC