| 00:03.14 | brlcad | hmm |
| 00:04.43 | brlcad | so we either git-annex them (to files hosted on brlcad.org) or put the repo itself on brlcad.org |
| 00:05.04 | brlcad | I like the idea of git-annex, never used it though |
| 00:05.58 | brlcad | or |
| 00:06.05 | brlcad | https://git-lfs.github.com/ |
| 00:06.38 | maths22 | brlcad: Trying irssi plubin for quassel |
| 00:06.41 | maths22 | It works! |
| 00:07.04 | maths22 | I'll look into those for the repository |
| 00:07.58 | brlcad | maths22: how does quassel deal with logging? I presume it does the logging... |
| 00:08.18 | maths22 | What logging? If you mean the back-log, I'm not really sure |
| 00:08.26 | brlcad | wonder how it compares with irssi's logging too |
| 00:08.33 | maths22 | see http://bugs.quassel-irc.org/projects/quassel-irc/wiki/quassel_logging |
| 00:09.58 | brlcad | ah, that sounds cool and sucky all at the same time |
| 00:10.08 | brlcad | looks like it stores all the chatter in a database |
| 00:10.36 | maths22 | I'm trying it out for a few days, and I'll see what I think |
| 00:10.46 | brlcad | and you have to manually dump to a file, instead of there being a backlog AND on-disk log that is always up-to-date |
| 00:10.53 | maths22 | The mobile support is particularly nice-I don't really like mobile ssh |
| 00:11.07 | maths22 | Regarding logging, this is part of why I am running it through an irssi proxy |
| 00:11.24 | maths22 | I still have my irrsi session up and logging as I always do, this just layers on top of it |
| 00:11.39 | brlcad | test what happens if you kill quassel-core .. if log data persists (I would expect it to as it says it's in an sqlite database) |
| 00:12.13 | maths22 | exit normally or force-kill? |
| 00:12.18 | brlcad | right, but running irssi from different locations on a client instead of the server means my logs are not in one place |
| 00:12.25 | brlcad | force-kill |
| 00:12.31 | brlcad | kill -9 |
| 00:12.43 | maths22 | brlcad: I'm saying I run irrsi on the server, and then connect to that from quassel |
| 00:13.07 | brlcad | oooh |
| 00:13.09 | brlcad | huh |
| 00:13.18 | brlcad | that would be kind of cool |
| 00:14.19 | maths22 | History persisted across restart |
| 00:15.17 | maths22 | I could still talk through my screen+irssi session-Quassel connects on top of that, so it going down doesn't result in me losing anything, including the ability to talk |
| 00:17.21 | maths22 | There exists a nice node-based client for it |
| 00:18.09 | maths22 | http://prntscr.com/9pt9oq |
| 00:18.40 | brlcad | sweet |
| 00:55.03 | *** join/#brlcad kintel (~kintel@unaffiliated/kintel) | |
| 01:26.16 | andromeda-galaxy | maths22: should I start running a downloader client locally? |
| 01:26.58 | maths22 | Not yet. I need to make some updates first |
| 01:28.29 | andromeda-galaxy | maths22: so, you'll update the java script and then send the updated version to me, so that I can run a download locally? I like the idea of doing that so that I can poke around and double-check that everything looks good, since more eyes on data are always better. |
| 01:28.52 | andromeda-galaxy | maths22: also: java? ouch... |
| 01:29.36 | andromeda-galaxy | maths22: if you don't mind, I might try rigging up a version of it with bash + jq for ufn |
| 01:29.37 | andromeda-galaxy | *fun |
| 01:29.50 | maths22 | I know java better than other languages, and didnt really know a better tool when I did it |
| 01:30.08 | maths22 | It actually isn't too bad |
| 01:30.37 | andromeda-galaxy | yeah, it's pretty well written. I don't suppose that I could convince you that we should do it in Haskell? |
| 01:33.19 | andromeda-galaxy | maths22: as I mentioned, I might mock something up in bash, just for fun though |
| 01:34.08 | maths22 | You could, but then I would need to refresh my Haskell memory :) |
| 01:34.30 | maths22 | Haskell version may happen tonight |
| 01:34.50 | maths22 | If I'm bored enough |
| 01:34.54 | andromeda-galaxy | maths22: I can probably do a Haskell or bash version tomorrow, unless something unforseen comes up |
| 01:35.28 | andromeda-galaxy | (bash is actually probably pretty simple: jq + some cmdline xml parser... honestly, that might be my first instinct for this problem) |
| 01:35.44 | maths22 | The java one is functionally fine, but rewiting is fun |
| 01:37.34 | andromeda-galaxy | maths22: indeed, that's the only reason I'm playing around with bash |
| 01:38.57 | andromeda-galaxy | maths22: by the way, why do you create a new url from the key, instead of using operations/row/link ? |
| 01:39.35 | maths22 | Which line are you referring to? |
| 01:40.08 | andromeda-galaxy | maths22: the way that you construct the per-task urls from the json from the main page |
| 01:40.27 | andromeda-galaxy | you use <url fragment> ++ id (parsed from the json) instead of the url in the json |
| 01:41.01 | maths22 | Probably the JSON format changed (or I wasn't paying much attention when I wrote it) |
| 01:41.10 | andromeda-galaxy | maths22: ah, okay |
| 01:41.17 | andromeda-galaxy | oops, it looks like I have to levae now suddenly, sorry |
| 01:41.20 | brlcad | since GCI has moved to a completely different system, remember that this is a ONE-SHOT deal.. doesn't matter what it's written in ;) |
| 01:41.24 | andromeda-galaxy | I might talk to you again a little later tonight.. |
| 01:41.34 | andromeda-galaxy | brlcad: of course! this discussion is just for fun |
| 01:41.42 | brlcad | nods |
| 01:42.15 | maths22 | I also have the unused variable "name" |
| 01:42.27 | brlcad | there are many other fun coding exercises that could use that attention too -- like the web interface that runs jobs on a checkout ;) |
| 01:42.59 | maths22 | brlcad: What are you referring to? |
| 01:44.00 | brlcad | we made it into a set of GCI tasks, I think andromeda-galaxy worked on it iirc |
| 01:44.42 | brlcad | basically, there's a pattern we seem to do a LOT on our code where we're either graphing some property over time or searching for when something happened, given some range of commits |
| 01:45.29 | brlcad | so we came up with a task idea to create a simple web interface where you provide a shell script, the range of commits, and it goes to town for you |
| 01:46.41 | brlcad | ideally automatically keeping track of the patching that you often need between broken commit versions, and optionally collapsing "series" of commits where the same author made N commits in a row around the same time |
| 01:47.33 | brlcad | but even a simple one that just runs the script on each commit would be useful without that |
| 01:56.17 | Notify | 03BRL-CAD:starseeker * 66866 (brlcad/trunk/include/fft.h brlcad/trunk/src/libfft/ditsplit.c brlcad/trunk/src/libfft/splitdit.c): A little doxygen rework on libfft. Still not at all well documented, but at least it's something... |
| 02:22.28 | Notify | 03BRL-CAD:starseeker * 66867 (brlcad/trunk/misc/doxygen/libbn.dox brlcad/trunk/misc/doxygen/libbu.dox): Slight clean-ups for libbu/libbn doxygen |
| 02:34.31 | Notify | 03BRL-CAD:starseeker * 66868 brlcad/trunk/CHANGES: list rle specific tools for deprecation |
| 02:34.51 | Notify | 03BRL-CAD:starseeker * 66869 brlcad/trunk/misc/doxygen/libbn.dox: shorten name |
| 02:38.46 | *** join/#brlcad zorryglguvddijgf (~armin@dslb-092-075-147-095.092.075.pools.vodafone-ip.de) | |
| 03:17.57 | andromeda-galaxy | brlcad, maths22: yeah, I still have code for that lying around somewhere (though I only got the "simple" version working before I ran out of time)... I'm still not sure what its security model should be, though |
| 03:56.17 | maths22 | brlcad: Web client dark screen http://prntscr.com/9pv0j6 |
| 04:05.02 | starseeker | maths22: out of curiosity, do you have any experience with review-board? https://www.reviewboard.org |
| 04:06.22 | starseeker | has been wondering if the newer versions can be teased into an effective post-commit review system for BRL-CAD, but as yet I've not been able to put all the pieces together... |
| 04:24.27 | *** join/#brlcad kintel (~kintel@unaffiliated/kintel) | |
| 04:27.41 | maths22 | starseeker: I have not heard of it |
| 04:27.59 | maths22 | I'll take a look tomorrow |
| 07:13.15 | *** join/#brlcad teepee_ (~teepee@unaffiliated/teepee) | |
| 07:50.42 | *** join/#brlcad gaganjyot (~gaganjyot@101.213.153.123) | |
| 09:00.59 | andromeda-galaxy | maths22: I think that I have a fairly decently robust implementation that properly handles multiple files with the same name and handles the fact that you can submit links as well as files in ~65 lines of bash. I've downloaded 2014 & most of 2013; the data dir size (uncompressed, with html versions of each task page + all task files) is 2.6 GB |
| 09:30.24 | *** join/#brlcad infobot (ibot@rikers.org) | |
| 09:30.24 | *** topic/#brlcad is http://ibot.rikers.org/%23brlcad/ || We're busy preparing a major release 7.26 (eta January) and getting archer/mged to Beta status (eta April) || SEEKING: web devs to finish setting up our new website! | |
| 09:59.24 | *** join/#brlcad teepee` (bc5c2134@gateway/web/freenode/ip.188.92.33.52) | |
| 10:17.07 | *** join/#brlcad merzo_ (~merzo@user-94-45-58-141.skif.com.ua) | |
| 10:34.41 | Notify | 03BRL-CAD Wiki:Seniadrian * 0 /wiki/User:Seniadrian: |
| 14:10.01 | *** join/#brlcad poxip (~poxip@unaffiliated/mrpoxipol) | |
| 14:24.12 | Notify | 03BRL-CAD:starseeker * 66870 brlcad/trunk/include/wdb.h: Doxygen tweaks for wdb.h |
| 14:27.30 | Notify | 03BRL-CAD:starseeker * 66871 (brlcad/trunk/include/rt/timer.h brlcad/trunk/misc/doxygen/librt.dox): Group the rt timer into librt doxygen. |
| 14:36.53 | Notify | 03BRL-CAD:starseeker * 66872 brlcad/trunk/misc/doxygen/libmultispectral.dox: Add a brief note about libmultispectral - need much better docs explaining all this... |
| 14:42.46 | Notify | 03BRL-CAD:starseeker * 66873 brlcad/trunk/misc/doxygen/CMakeLists.txt: reorder libraries, add a note that this order is important to doxygen and not arbitrary. |
| 14:48.12 | Notify | 03BRL-CAD:starseeker * 66874 (brlcad/trunk/misc/doxygen/libbg.dox brlcad/trunk/misc/doxygen/libbn.dox and 2 others): Don't library names in definition labels - context makes it clear. |
| 15:02.04 | *** join/#brlcad kintel (~kintel@unaffiliated/kintel) | |
| 15:07.57 | Notify | 03BRL-CAD:ejno * 66875 (svn:ignore ## -0,0 +1,3 ## and 3 others): create a branch for work on extracting NMG to a library |
| 15:11.28 | andromeda-galaxy | brlcad, maths22: I have 3.1GB uncompressed of data. Also, be warned that the Melange-generated pages aren't valid (x)html, in some cases they even have invalid unicode characters |
| 15:11.58 | maths22 | Not surprising. Did you unzip the files, or are they still in the zip format from when they were downloaded? |
| 15:12.51 | andromeda-galaxy | maths22: curl --retry 3 -s "http://www.google-melange.com/$link" | gunzip - > "$file" # line from the script |
| 15:12.54 | andromeda-galaxy | so yes |
| 15:13.20 | maths22 | OK. Could you share the script with me? I would like to see it |
| 15:14.00 | andromeda-galaxy | brlcad: if you have a list of categories to file the tasks into, I can make a tiny little script to make refiling them easier |
| 15:14.07 | andromeda-galaxy | maths22: sure! how do you want me to? |
| 15:14.16 | maths22 | pastebin? |
| 15:16.25 | andromeda-galaxy | maths22: sounds good, hold on one second |
| 15:17.53 | andromeda-galaxy | maths22: http://lpaste.net/2293901509777162240 |
| 15:18.38 | maths22 | andromeda-galaxy: Thanks! |
| 15:19.08 | andromeda-galaxy | maths22: no problem. There were some particularly annoying problems with Melange though: |
| 15:19.22 | *** join/#brlcad kintel (~kintel@unaffiliated/kintel) | |
| 15:20.57 | andromeda-galaxy | The main one was that for some reason, one html file included an ASCII code 5 "ENQ" character, which isn't legal in xml. I used to use hxnormalize instead of xmlstarlet fo on like #26, but that passed the ENQ character through, and the xmlstarlet sel invocation then errored out, and no files were downloaded. Luckily, I have an assertion that, through the complexities of wc, ensures that the number of files found, |
| 15:20.57 | andromeda-galaxy | was downloaded, so I caught it. |
| 15:21.09 | andromeda-galaxy | that'll be important if you take a similar approach on any other scripts |
| 15:22.01 | maths22 | interesting |
| 15:22.08 | andromeda-galaxy | maths22: fun idea: extract the comments, intersperse them with links to the downloaded file, and then run the whole thing throuph pandoc to get a nice little markdown document for each task with task title, task description, and files interspersed with comments |
| 15:22.52 | andromeda-galaxy | (e.g. use pandoc to build the markdown from html) |
| 15:23.29 | andromeda-galaxy | currently, the tasks are written as ${year}/${org}/${task-key}/files/${date-uploded}/${filename} |
| 15:23.34 | andromeda-galaxy | or, if that's ambiguous, |
| 15:23.47 | andromeda-galaxy | ${year}/${org}/${task-key}/files/${date-uploded}/${filename}/$n where n = [0..] |
| 15:23.51 | andromeda-galaxy | s/0/1 |
| 15:24.00 | andromeda-galaxy | maths22: does that seem like a deceent scheme to you? |
| 15:24.20 | maths22 | It does seem reasonable |
| 15:24.58 | andromeda-galaxy | oh good. Now we have to figure out what to do with 3.1 GB of task data |
| 15:27.23 | maths22 | zip it up and upload it to transfer.sh |
| 15:28.05 | andromeda-galaxy | maths22: sure, should I try generating the abovementioned markdown summaries quickly? |
| 15:28.14 | maths22 | Sounds reasonable |
| 15:29.10 | andromeda-galaxy | maths22: great, I'll try to do that quickly, |
| 15:29.32 | andromeda-galaxy | after I send it to transfer.sh, are we planning that you'll double-check that everything looks good & we'll stick the files on brlcad.org (so that we can set up the git annex repository)? |
| 15:29.39 | maths22 | yep |
| 15:31.04 | andromeda-galaxy | maths22: sounds good. Do you think that we should we upload all the files (including task pages and Markdown summaries) or just the large files? |
| 15:31.35 | maths22 | Probably all of them, but once we have the zip on the server, we can play with it however we want |
| 15:31.56 | Notify | 03BRL-CAD:ejno * 66876 (brlcad/branches/libnmg/include/nmg.h brlcad/branches/libnmg/src/CMakeLists.txt and 23 others): remove rt includes and separate rt-dependent code (in progress) |
| 15:33.29 | andromeda-galaxy | wait, are we going to put a zip on the server? I don't think that git-annex will be able to deal with that... |
| 15:33.48 | maths22 | I'll unzip it once it is there |
| 15:34.16 | andromeda-galaxy | maths22: good, just checking |
| 15:34.33 | andromeda-galaxy | are you planning on handling everything that needs access to the server? |
| 15:36.49 | andromeda-galaxy | (I've never played with it before, and thus don't have any access to it) |
| 15:38.47 | *** join/#brlcad kintel (~kintel@unaffiliated/kintel) | |
| 16:01.10 | *** join/#brlcad kintel (~kintel@unaffiliated/kintel) | |
| 16:17.44 | Notify | 03BRL-CAD:ejno * 66877 (brlcad/trunk/src/conv/gcv/gcv.c brlcad/trunk/src/libgcv/gcv.c and 2 others): add a decimation filter |
| 16:40.54 | Notify | 03BRL-CAD:ejno * 66878 brlcad/trunk/src/libgcv/plugins/decimate/decimate.c: print decimation statistics |
| 16:52.39 | maths22 | andromeda-galaxy: Indeed |
| 16:57.21 | andromeda-galaxy | maths22: great! I have a few other things to do this morning, but I should have an archive with the Markdown summaries for you in a few hours at most |
| 16:57.37 | maths22 | Ok. Sounds good. No hurry |
| 16:58.24 | andromeda-galaxy | also: what do you think of using ${year}/${org}/${task-key}/${date}/${filename}/${n}/${filename} for the tasks with more than one copy of a filename at a given date? (previously, it ended at /${n}); that way, tools that rely on file extensions will still work |
| 17:00.01 | maths22 | This is a very good idea (or you could rename to ${filename}.${n}.${ext}) |
| 17:01.12 | andromeda-galaxy | maths22: the reason that I was planning on continuing to use directories was that I figured that it might otherwise be ambiguous whether somebody submitted ${basename}.0.${ext} and ${basename}.1.${ext}, or if they submitted ${basename}.${ext} twice... |
| 17:01.34 | maths22 | That is reasonable |
| 17:02.19 | andromeda-galaxy | excellent, I'll rework the script & inplace update all the data before sending it to you |
| 17:08.16 | *** join/#brlcad kintel (~kintel@unaffiliated/kintel) | |
| 17:08.16 | andromeda-galaxy | maths22: another possibility (which would keep more ordering information) is ${date}/${n}/${filename}, with $n uniformly increasing throughout the tasks... thoughts on benefits/tradeoffs of that approach? |
| 17:08.58 | maths22 | I actually like that idea. What again is the reason for the date folder? |
| 17:09.14 | Notify | 03BRL-CAD:starseeker * 66879 (brlcad/trunk/CHANGES brlcad/trunk/include/CMakeLists.txt): Remove conf.h - has been deprecated since 7.0 |
| 17:10.50 | andromeda-galaxy | maths22: just to hold on to the information about when it was submitted (for ordering purposes, injecting into the comment stream, etc.) |
| 17:11.30 | maths22 | Shouldn't that be encoded into the file's creation date? |
| 17:12.17 | andromeda-galaxy | maths22: hmm, that's an interesting idea... |
| 17:12.39 | andromeda-galaxy | by the way, I just realized that I meant to say throughout the task---e.g. the 50th task downloaded wouldn't start with n=320 |
| 17:13.10 | andromeda-galaxy | maths22: the advantage of the folder structure (to me) is that it's less "fragile" than the dates---any number of archival methods modify the various dates on the file, |
| 17:13.46 | andromeda-galaxy | including, IIRC, git. In addition, if we end up moving tasks around between directories a lot to manage a queue, making sure that everyone who checks out the repo has everything set up right so that creation dates get preserved is a big pain |
| 17:14.48 | maths22 | That is fair; I just feel like one should be able to list a single directory and find all the files assosciated with the task (or n folders with one file per folder) |
| 17:15.08 | andromeda-galaxy | maths22: I was assuming that people wouldn't mind find $task/$files -type f |
| 17:15.19 | maths22 | andromeda-galaxy: I always forget about find :( |
| 17:15.20 | maths22 | Never ind |
| 17:15.25 | maths22 | s/ind/mind/ |
| 17:15.45 | andromeda-galaxy | maths22: oh okay. so are we agreed on ${date}/${n}/${file}, with $n starting at 1 and counting up through all the files for a given date? |
| 17:15.55 | maths22 | yes |
| 17:16.05 | andromeda-galaxy | maths22: great, I'll rework the scripts |
| 17:16.50 | andromeda-galaxy | maths22: I just got a test of the markdown conversion going though, and it's a little worrying: pandoc doesn't seem to do a very good job, probably because it doesn't read any css and the comments are all horribly nested divs |
| 17:16.58 | andromeda-galaxy | so I'm probably going to have to do markdown by hand :( |
| 17:17.31 | maths22 | That's dissapointing |
| 17:18.14 | andromeda-galaxy | yeah, now I'm trying to decide between 1. making html summaries and no text-readable ones, 2. making markdown summaries through text concatenation, and 3. doing both of the above |
| 17:18.54 | andromeda-galaxy | part of the problem is that I'm actually not at all sure how to get the markdown to look any good, since we need to put a number of properties under each comment heading |
| 17:19.12 | Notify | 03BRL-CAD:starseeker * 66880 (brlcad/trunk/include/optical/light.h brlcad/trunk/include/optical/shadefuncs.h brlcad/trunk/include/optical/shadework.h): Add liboptical headers to liboptical group |
| 17:20.43 | andromeda-galaxy | maths22: you get markdown like: |
| 17:20.49 | andromeda-galaxy | div id="c" class="single-comment"><span class="single-comment-author">[Marc Tannous](/gci/student_tasks/google/gci2014/tannousmarc)</span><span class="single-comment-meta">on December 1 2014 17:03 UTC</span><span class="single-comment-title">Task Claimed</span> |
| 17:20.51 | andromeda-galaxy | which is reallyjust html |
| 17:20.57 | Notify | 03BRL-CAD:starseeker * 66881 (brlcad/trunk/include/dm/bview.h brlcad/trunk/include/vmath.h brlcad/trunk/misc/doxygen/CMakeLists.txt): Add the two clearly public headers that are not (at the moment) associated with a library. |
| 17:20.59 | maths22 | This is not markdown :( |
| 17:21.21 | andromeda-galaxy | exactly |
| 17:35.33 | Notify | 03BRL-CAD:ejno * 66882 brlcad/trunk/src/libgcv/gcv.c: remove the initial tessellation filter for now, until it works correctly |
| 17:51.52 | andromeda-galaxy | maths22: I've got to leave now, but if you have any thoughts on good ways to do a presentation of the data while avoiding that problem, let me know |
| 17:52.09 | maths22 | I'll take a look later |
| 18:24.25 | *** join/#brlcad ickby (~stefan@x5d84c445.dyn.telefonica.de) | |
| 18:42.52 | Notify | 03BRL-CAD:n_reed * 66883 brlcad/trunk/doc/STRATEGY: renamed strategic areas and converted TODOs to org-mode headlines |
| 19:10.13 | *** join/#brlcad teepee_ (~teepee@unaffiliated/teepee) | |
| 19:42.32 | Notify | 03BRL-CAD:starseeker * 66884 (brlcad/trunk/CHANGES brlcad/trunk/include/bu/cmd.h brlcad/trunk/src/libtclcad/cmdhist_obj.c): Deprecated the bu_cmdhist functionality as public libbu API. Need to decide what libged level functionality we'll eventually want, for now provide tclcad the logic internally. |
| 19:59.02 | brlcad | andromeda-galaxy: responding to your earlier comment, the design gallery page has a list of the design categories of interest (plus icons or misc for any remaining design tasks), plus 'code', 'docs', and probably 'infrastructure' or 'etc' ... we can subdivide |
| 19:59.32 | brlcad | if it can be done hierarchically, that would be cool, to just start with "design", "code", "docs", and "etc", see what that looks like |
| 20:00.40 | brlcad | I see your shell scripting skills have improved considerably too :) |
| 20:01.26 | andromeda-galaxy | brlcad: sounds good, I'll try to set it up heirarchically like that |
| 20:01.54 | andromeda-galaxy | brlcad: yeah, I got a lot better at bash after writing bash completion for ovs-vsctl (from Open vSwitch)---that's a really complex command, so we used a few hundred lines of bash |
| 20:02.35 | brlcad | fyi, we don't need (or want) year or org delineation other than to maybe order integration better .. it's all categoric work that has to be processed |
| 20:02.45 | brlcad | the task-key should be chrono hopefully |
| 20:03.20 | andromeda-galaxy | we can probably get rid of that at some point, but I left it in to the script to make sure that we don't accidentally overwrite things, since I have absolutely no idea how the task keys are generated---the 2014 and 2013 ones both look like random strings of numbers to me, |
| 20:03.27 | andromeda-galaxy | although 2012 is slightly shorter random strings |
| 20:04.08 | andromeda-galaxy | I'm trying to figure out what sort of "task summary" file we want to give someone opening a new task, I originally thought that I' d just render an html page with comments & inline links to the files that we scraped to markdown (so that people can look at it in an editor), but it looks like that's somewhat difficult, since html->markdown isn't very good |
| 20:04.21 | andromeda-galaxy | also: xmlstarlet & jq are awesom |
| 20:04.26 | andromeda-galaxy | s/$/e |
| 20:07.22 | brlcad | still reading backlog, liked maths22 notion that we should be able to list a single dir to find all the files / folders for a given task instead of date nesting but only because the date is pretty much irrelevant for integration purposes |
| 20:08.22 | brlcad | yay, caught up |
| 20:08.45 | andromeda-galaxy | brlcad: that was about the setup for inside one task, though---I thought that the date would be relevant for while a contributer was trying to figure out which files to actually use |
| 20:09.15 | brlcad | you're right for the script, it needs to be careful, just saying it's of no use for the repo structure |
| 20:09.46 | andromeda-galaxy | also, if we go the git annex route, we can just stuff all of the data up on brlcad.org in a relatively unstructured format and then move around annex's marker files in the repo, right? |
| 20:09.47 | brlcad | what's the $n ? |
| 20:09.51 | Notify | 03BRL-CAD:starseeker * 66885 (brlcad/trunk/include/bio.h brlcad/trunk/include/bnetwork.h and 3 others): Doxygen work for bio/bnetwork |
| 20:10.01 | andromeda-galaxy | in case there's more than one file for the same datetime with the same name |
| 20:10.09 | andromeda-galaxy | the time resolution is only good to within a minute, so that really does happen |
| 20:10.53 | brlcad | it's not a big deal really either way .. if someone is processing a task, they will almost certainly look at all the files |
| 20:11.08 | brlcad | so the easier it is to look at all of them, the faster it will be to et through 795 tasks |
| 20:11.14 | brlcad | s/et/get/ |
| 20:11.24 | andromeda-galaxy | (I was thinking that we should just upload files with the maximum of structure---it seems like this kind of repetitive structure will be easier to get rid of than to add back in) |
| 20:11.37 | brlcad | knowing the submission order is obviously helpful for knowing what came before/after what |
| 20:12.52 | andromeda-galaxy | we could also do ${task-key}/files/${n}/${filename}, cutting out the ${date} folders but keeping the ordering in the ${n}... I worry a bit about putting all of them in the same directory, though, it seems like it would be too easy to overwrite older files |
| 20:13.03 | brlcad | can do that, but then there will almost certainly be up-front work to immediately break it down to something minimal |
| 20:13.22 | andromeda-galaxy | (well, ${task-key}/files/${n}-${filename} might work too) |
| 20:13.22 | brlcad | this isn't surviving post integration, it's all about making them as fast as possible to review |
| 20:14.19 | andromeda-galaxy | I just wanted to download files with enough structure to put it all back in if we want it, a few lines of bash sshould be enough to move things back around |
| 20:14.43 | andromeda-galaxy | I see your point about one directory now, though---lots of tools are conducive to "open everything in this directory" |
| 20:14.44 | brlcad | what about ${title}-${year}-${task-key}/${n}-${filename} |
| 20:15.13 | brlcad | or something similar ... really flat hierarchy but guaranteed to not conflict |
| 20:15.18 | andromeda-galaxy | that's nice |
| 20:15.24 | brlcad | and guaranteed to order correctly |
| 20:15.36 | maths22 | Title should then be at the end |
| 20:15.41 | andromeda-galaxy | encoding ${title} might be kind of annoying though, since we have #'/etc. in there |
| 20:15.42 | brlcad | sure |
| 20:16.01 | brlcad | could just strip non-alphanum chars |
| 20:16.09 | andromeda-galaxy | e.g. it's easy to just use "" and stick any random set of characters in there, but it might annoy people later |
| 20:16.14 | andromeda-galaxy | oh that's true |
| 20:16.26 | andromeda-galaxy | sed s/[^A-Za-z0-9]//g |
| 20:17.05 | andromeda-galaxy | or tr -cd '[[:alnum:]]' |
| 20:17.18 | brlcad | yep |
| 20:18.10 | andromeda-galaxy | or actually, we might want to replace spaces with dashes first, but anyway |
| 20:18.14 | Notify | 03BRL-CAD:starseeker * 66886 (brlcad/trunk/include/bsocket.h brlcad/trunk/misc/doxygen/CMakeLists.txt brlcad/trunk/misc/doxygen/libbu.dox): handle bsocket like bio and bnetwork |
| 20:18.35 | brlcad | sed 's/[^[:alnum:]_-]//g |
| 20:18.55 | brlcad | underscores probably, but yeah |
| 20:19.36 | brlcad | so we end up with something like: |
| 20:19.50 | andromeda-galaxy | we might want a /files in there after task before ${n}-*, though |
| 20:19.59 | andromeda-galaxy | that way we can stick the task description in there, and have a different folder for links |
| 20:20.27 | andromeda-galaxy | (since students can submit links to work online, or to websites related to the task in any way, and we probably don't want to accidentally crawl too much) |
| 20:20.46 | brlcad | docs/2013_8234829392_Convert_to_Doxygen/178218383-file1.doxygen |
| 20:20.47 | andromeda-galaxy | right now, those get done as ${task-key}/links/$n, with $n monotonically increasing for the task |
| 20:21.17 | andromeda-galaxy | where'd the 178218383 come from? are you talking about using the id from the html file instead of a new counter? |
| 20:22.09 | brlcad | doesn't matter, as long as it's ordered somehow |
| 20:22.16 | andromeda-galaxy | are the ids ordered? |
| 20:22.17 | brlcad | could be the date or counter |
| 20:22.49 | brlcad | I assumed they were based on what I thought I saw when the program started, but don't know |
| 20:22.55 | andromeda-galaxy | well, date doesn't work, since there could be duplicates; we could do ${date}-${counter} or just ${counter} (which seems simpler, and doesn't lose anything since we still have task html file |
| 20:23.44 | brlcad | the discussion in the html file, though, is chrono, so won't really know what file is being talked about without a date somewhere |
| 20:24.37 | brlcad | ${n}-${date}-${filename} sounds pretty good |
| 20:24.58 | andromeda-galaxy | yeah, true, though to some degree that goes back to what I was saying earlier about providing a summary file in some human friendly format |
| 20:25.26 | andromeda-galaxy | (${n}-${date} would have ${n} increasing throughout the whole task?) |
| 20:25.42 | brlcad | yeah |
| 20:26.26 | andromeda-galaxy | okay, that sounds like a good plan: ${category}/${year}-${key}-${title}/files/${n}-${date}-${filename} |
| 20:26.30 | andromeda-galaxy | (for the files) |
| 20:26.42 | brlcad | doesn't matter how, as long as it's easy to tell what file came before what others and that map to the conversation/html, we'll be good |
| 20:26.45 | andromeda-galaxy | ${category}/${year}-${key}-${title}/task.html for the task descr |
| 20:26.48 | brlcad | I assume you're not doing html re-writing |
| 20:27.04 | andromeda-galaxy | and ${category}/${year}-${key}-${title}/links/${n}-${date} for the links |
| 20:27.19 | andromeda-galaxy | brlcad: that's what I was discussing with maths22 earlier, |
| 20:27.39 | andromeda-galaxy | at this point task.html is just a cleaned-up copy of the Melange page (using xmlstarlet fo --recover to clean it up) |
| 20:27.55 | andromeda-galaxy | but it would be nice to provide a summary with links to the local copies of the file |
| 20:28.05 | andromeda-galaxy | but I don't know if html (which would be easy to do using xmlstarlet) is the best idea or not |
| 20:28.35 | andromeda-galaxy | since that would require one to open a web browser for each task to look at a nicely formatted version of the page |
| 20:28.41 | brlcad | or even ${n}-${date}-${link} |
| 20:28.51 | Notify | 03BRL-CAD:n_reed * 66887 brlcad/trunk/doc/STRATEGY: convert cross-over designations to org-mode tags |
| 20:29.22 | brlcad | if we can avoid the subdir and get everything in one dir, it will be that many cd's fewer * 795 |
| 20:29.31 | Notify | 03BRL-CAD:n_reed * 66888 brlcad/trunk/doc/STRATEGY: fix typo |
| 20:29.48 | brlcad | how are you handling the urls? |
| 20:30.03 | brlcad | just embedding them in a text file, or the name of the file is the url? |
| 20:30.13 | brlcad | or creating an html with that link in it? |
| 20:30.21 | brlcad | something else? |
| 20:30.35 | andromeda-galaxy | brlcad: right now, lins/${date}-${n} is a plain text file with one line containing the value of the href attribute of the a element which generated the link |
| 20:30.45 | andromeda-galaxy | since ${n} |
| 20:31.09 | brlcad | cool, that works |
| 20:31.24 | andromeda-galaxy | is monotonically increasing, we could write all the links to files named ${n}-${date}-link and hope that no-one confuses them with files, since we probably won't have ${filename}==link |
| 20:32.12 | andromeda-galaxy | (by the way, links are currently detected by finding <a> elements with rel=nofollow, there doesn't seem to be anything else to distinguish them, except that they aren't to a subsite of google-melange.com) |
| 20:36.47 | andromeda-galaxy | brlcad: also, what do you think of providing a file like this for a "landing page" for each task: http://lpaste.net/3413613330818924544 |
| 20:37.22 | andromeda-galaxy | (the names & first few times are from a real datafile, but I embellished the end & avoided copying in the description) |
| 20:38.32 | andromeda-galaxy | the main problem is that I'm not sure how to actually get markdown (or similar) versions of the task description and comment texts, pandoc does a horrible job with it... |