Irc discussions from #mirage on 24-05-2017

Written by lobo
Classified under: irclog
Published: 2017-06-25 (last updated: 2017-06-25)

25-05-2017 15:03 yomimono rushes into room

25-05-2017 15:03 yomimono hi folks!

25-05-2017 15:03 avsm greetings!

25-05-2017 15:03 TImada Hi

25-05-2017 15:03 lobo hi

25-05-2017 15:03 yomimono we said we'd have an IRC meeting today!

25-05-2017 15:04 yomimono so let's do that!

25-05-2017 15:04 yomimono the agenda is at https://github.com/mirage/mirage-www/wiki/Call-Agenda and it's looking very sparse; please add anything you'd like to discuss

25-05-2017 15:04 avsm i am penning my update as we speak :-)

25-05-2017 15:05 Drup My only entry is "Please help review https://github.com/ocsigen/lwt/pull/354 \o/"

25-05-2017 15:05 yomimono while you're doing that, I'll start us off - there's now a 4.04.0+afl compiler available from the default opam repository

25-05-2017 15:06 yomimono drup: ooh, that's a good one

25-05-2017 15:06 avsm oh stedolan helped me with afl, and i got test cases i am going to submit for ocdaml-hex (which was the simplest repo i could come up with try learning it)

25-05-2017 15:06 mato waves, distractedly (setting up dad with a new phone, 1st time he's seen android, blah...)

25-05-2017 15:07 yomimono I would really recommend that folks try out crowbar (https://github.com/stedolan/crowbar) on stuff, the reward to effort ratio is amazing

25-05-2017 15:08 Drup I have one question with this kind of stuff: how do you do this kind of autogenerated testing thingy when you have two input that should coincide ?

25-05-2017 15:08 hannes I fixed my cycle bug in my DNS resolver last night... will writeup soon

25-05-2017 15:08 Drup Like, let's say, a regular expression library

25-05-2017 15:08 avsm hannes: dns would an amazing target for afl i think -- crowbar has an examples/ directory with some similar ones

25-05-2017 15:08 Drup you want to generated regular expression, and also positive and negative inputs

25-05-2017 15:09 avsm im going to submit ocaml-hex first (sadly no bugs found) and then expand out to cstruct

25-05-2017 15:09 Drup and I'm just not sure how that would work

25-05-2017 15:09 yomimono drup: by positive and negative inputs you mean should-match, should-not-match ?

25-05-2017 15:09 Drup yes

25-05-2017 15:09 yomimono yeah, I'm having trouble imagining how you'd do that without another regex implementation to check your work against

25-05-2017 15:10 Drup Yeah, you sort of need an oracle

25-05-2017 15:10 hannes avsm: yes, I already used hours of afl-persistent for my udns library. crowbar is still on my todo

25-05-2017 15:11 yomimono hannes: will be cool to see the writeup on that :)

25-05-2017 15:11 hannes there won't be many good words on ocaml-dns, unfortunately... sorry

25-05-2017 15:12 yomimono I think you can just link to a recent issue in that repository ;)

25-05-2017 15:12 yomimono which lays out some good ideas for improvement IIRC

25-05-2017 15:12 Drup every time I get interested into testing things, I bump into this kind of situations where I have two inputs and I'm not sure how generate test cases

25-05-2017 15:12 avsm hannes: very cool

25-05-2017 15:13 hannes avsm: btw, it would be really nice if you could spend 10 minutes on https://github.com/mirage/ns.mirage.io/issues/1 ... the zone can't be that big that to copy + paste the important bits ;)

25-05-2017 15:13 avsm Drup: it's worth creating an issue to ask this on crowbar

25-05-2017 15:14 yomimono +1

25-05-2017 15:14 avsm stedolan has some examples

25-05-2017 15:14 avsm hannes: sorry, i never have my tfa device with me when it comes up... its at home right now by mistake ;-)

25-05-2017 15:15 hannes that's why two-factor auth is useless ;P

25-05-2017 15:15 yomimono the crowbar-y way to do this is use one set of random input to generate both the regex and the input, based on some rule that should generate a positive or negative case, but in this example that basically ends up being another regex engine embedded in your test code

25-05-2017 15:15 yomimono stedolan probably has better ideas than I do :P

25-05-2017 15:16 hannes and off to the wheeler lecture...

25-05-2017 15:16 yomimono waves

25-05-2017 15:16 avsm i am planning to try to build in google clusterfuzz support as well

25-05-2017 15:16 avsm it appears to be a single C callback

25-05-2017 15:17 avsm and stedolan reckons it should work pretty well

25-05-2017 15:17 avsm google clusterfuzz is a hosted service, so it would run on an ongoing basis for finding the really hard to reach places (and it supports AFL from first glance)

25-05-2017 15:18 avsm anyone interested in that, do get in touch

25-05-2017 15:18 yomimono that would be great to have working, please update the issue on mirage/mirage with your progress :)

25-05-2017 15:18 avsm yep will do

25-05-2017 15:18 yomimono :D

25-05-2017 15:18 avsm (it's a background item for me, so no guarantees about time :P

25-05-2017 15:18 yomimono anything else on that topic?

25-05-2017 15:18 yomimono oh! actually I do -

25-05-2017 15:19 yomimono I made some PRs in mirage-canopy-data including a new project for tcpip testing

25-05-2017 15:19 yomimono I would really appreciate review on those

25-05-2017 15:19 yomimono I merge a lot of my own PRs and I would like to stop doing that :)

25-05-2017 15:19 yomimono canopy-data is the repository that hosts content for canopy.mirage.io , you can see outstanding PRs for it at https://github.com/mirage/canopy-data/pulls

25-05-2017 15:21 yomimono done for real with that now, maybe?

25-05-2017 15:21 avsm aha

25-05-2017 15:21 avsm https://github.com/mirage/ocaml-hex/pull/20

25-05-2017 15:21 avsm need to figure out some scheme to automate the afl-fuzz invocations

25-05-2017 15:21 avsm so some work in that direction as we get more tests is probably useful too

25-05-2017 15:22 avsm it doesnt quite fit in with build-test in OPAM -- perhaps we need a build-fuzz in opam2 that uses the right afl compiler dependency and so on

25-05-2017 15:22 avsm not thought about it entirely yet

25-05-2017 15:22 yomimono generally speaking, shims from stdin to various device module types will probably be useful for mirage too

25-05-2017 15:23 avsm yes!

25-05-2017 15:24 avsm im all done with afl now :P

25-05-2017 15:24 yomimono OK :P

25-05-2017 15:24 yomimono next is an item I added on drup's behalf: Please help review https://github.com/ocsigen/lwt/pull/354 \o/

25-05-2017 15:24 yomimono this is a PR that rewrites lwt.ml and improves readability drastically

25-05-2017 15:25 yomimono having bounced off the old lwt.ml a few times, I'm pretty excited about this

25-05-2017 15:25 Drup Me too, But I really don't have time :(

25-05-2017 15:25 avsm Likewise, I am psyched to see this but havent had time for a proper review

25-05-2017 15:25 avsm perhaps we should just pin it locally and see what explodes (spoiler: nothing will!)

25-05-2017 15:26 yomimono having an understandable lwt.ml will help new contributors to mirage too, so if nothing else give a thumbs-up for the work :)

25-05-2017 15:27 Drup Just reading the new file is already quite helpful

25-05-2017 15:27 smondet At the Compose unconference/hackaton, Anton went through the lwt.ml resulting of this PR in front me and other old-ocamlers, I can say I was definitely convinced this is a good thing.

25-05-2017 15:28 avsm awesome!

25-05-2017 15:28 avsm i have made a comment on the tiny little diff in _tags instead of lwt.ml. starting easy

25-05-2017 15:29 yomimono anything more on that?

25-05-2017 15:29 Drup while we are on lwt, and to transition with the next topic

25-05-2017 15:30 Drup https://github.com/ocsigen/lwt/pull/374

25-05-2017 15:30 yomimono nice!

25-05-2017 15:31 avsm OMG I didnt realise that had happened! ^^ djs55

25-05-2017 15:31 yomimono (for the link-averse, that's a port of lwt to jbuilder by andyray)

25-05-2017 15:31 avsm So I just mailed the devel list as well

25-05-2017 15:31 avsm but the good news is that batch-building of multiple repositories seems to just work

25-05-2017 15:32 djs55 very impressive!

25-05-2017 15:32 avsm mkdir mybuild cd mybuild git clone -b jbuilder git://github.com/avsm/mirage-fs git clone -b jbuilder git://github.com/avsm/mirage-fs-unix git clone -b jbuilder git://github.com/avsm/mirage-clock git clone -b jbuilder git://github.com/avsm/mirage-device git clone -b jbuilder git://github.com/avsm/mirage-kv jbuilder build jbuilder runtest

25-05-2017 15:32 avsm oops

25-05-2017 15:32 avsm https://github.com/mirage/mirage/issues/818#issuecomment-303756041

25-05-2017 15:32 yomimono curb your enthusiasm, sir

25-05-2017 15:32 avsm has the commands

25-05-2017 15:32 avsm i am beside myself with dopamine overdose

25-05-2017 15:32 avsm with sufficient jbuild integration, i can come up with an opam variant that calls 'jbuilder' just ones

25-05-2017 15:33 avsm but i think this goes a long way to streamlining mirage's growing repository problem

25-05-2017 15:33 avsm we should never have to have a repeat of the difficulties of the mirage-3 api evolution, hopefully

25-05-2017 15:33 avsm since once we finish porting, we can test revdeps with a single jbuilder invocation

25-05-2017 15:33 avsm and fix breakage as we go along

25-05-2017 15:33 avsm this is very exciting!

25-05-2017 15:34 djs55 a script (or opam subcommand) which could check out the right versions of the sources of the dependencies for a project would be very helpful (like Go's "vendoring")

25-05-2017 15:34 avsm yes, I'll work on that

25-05-2017 15:34 avsm in the meanwhile, we need to port the remaining libraries on http://docs.mirage.io to jbuilder

25-05-2017 15:34 avsm it also has the benefit of respecting the odig version convention

25-05-2017 15:34 avsm although i think a 'jbuilder subst' is needed to add the version information djs55 -- that might be missing from your instructions

25-05-2017 15:34 avsm (or i might have been lazy and not followed them, unsure which)

25-05-2017 15:35 Drup djs55: isn't that pretty much "for f in {...}; opam source $f;" ?

25-05-2017 15:35 djs55 I probably forgot to write that

25-05-2017 15:35 avsm https://mirage.io/wiki/packaging

25-05-2017 15:35 avsm is djs55's instructions

25-05-2017 15:36 avsm https://github.com/mirage/mirage-clock/pull/30/files#diff-7122b1f0a3d7f01b665067bbc2939e3f -- this one needs some review

25-05-2017 15:36 avsm we are hitting some of the more complex packages now, and so i am also trying the Configurator config system

25-05-2017 15:36 avsm it works pretty well -- the configure test is only run as a dependency of the build, so its not built unless necessary (which is nice)

25-05-2017 15:36 djs55 Drup: I guess if I had a record of the package versions (the output of the solver?) and could check those out and untar/unzip them in the right place I'd be very happy. Maybe opam does this already… I'm not an advanced user :)

25-05-2017 15:37 Drup ah, right

25-05-2017 15:37 avsm i suspect some combination of opam install --show-actions and opam source is sufficient to get this

25-05-2017 15:37 Drup I think this was the original goal of opam bundle, sort of

25-05-2017 15:38 avsm yeah

25-05-2017 15:38 avsm i added a late agenda item about https://discuss.ocaml.org

25-05-2017 15:38 avsm it would be nice to share jbuilder porting tips (and other stuff ,like opam and opam-ed) on there

25-05-2017 15:39 yomimono you want to talk about cstruct first?

25-05-2017 15:39 avsm its a new experiment in the community, and so i wanted to check if that was a good place for us to start swapping mirage library management methods

25-05-2017 15:40 avsm sure -- cstruct is very boring though. I just wanted to say that https://github.com/ocaml/opam-repository/pull/9154 is progressing and touches a zillion packages. if you spot breakage please let me know

25-05-2017 15:40 avsm oh and i got so bored i also posted about using opam-ed to automate the constraint updates: https://discuss.ocaml.org/t/opam-ed-quickly-edit-opam-fields-from-the-cli/257

25-05-2017 15:41 avsm back to discuss.ocaml.org, in a smooth segway :-P

25-05-2017 15:43 avsm anyone got opinions about using this more?

25-05-2017 15:43 avsm or are you all too busy posting there to comment here

25-05-2017 15:43 avsm i am assuming the latter

25-05-2017 15:43 yomimono I was too busy asking my Twitter feed for strategies to deal with stuff that lives in the browser and wants to be occasionally checked

25-05-2017 15:44 yomimono Loads of people like interacting with things that are like that, so there must be a good way of doing it

25-05-2017 15:44 yomimono I'll go on a fact-finding mission and report back

25-05-2017 15:44 avsm hah yes indeed. working together to figure out how to make it more integrated with email workflow would be useful. https://discuss.ocaml.org/t/email-subscription/137/

25-05-2017 15:44 Drup yomimono: in the case of discuss, mailing list mode works fairly well

25-05-2017 15:45 avsm that's all i had to say on that really -- encourage people to use the forum to see if we can make it a more central place for the ocaml-related parts of mirage

25-05-2017 15:45 avsm ill mail the devel list as well

25-05-2017 15:45 avsm i think it also solves a lot of problems with newcomers to mirage -- the email list can be a little intimidating due to the difficulty of browsing archives

25-05-2017 15:45 yomimono yeah, and we've had people report that their messages go missing :(

25-05-2017 15:46 avsm thats all from me on that

25-05-2017 15:46 djs55 I've also forgotten the moderator password after the last change :(

25-05-2017 15:47 avsm haha

25-05-2017 15:47 yomimono that... seems bad, are you the only person who had a moderator account?

25-05-2017 15:47 avsm i think a gentle retirement of the mailing list, and/or a migration towards a @mirage.io for it makes sense anyway

25-05-2017 15:47 Drup yomimono: in general, my strategy is the following: "Does it has RSS ? Yes, use RSS. No: Do I care enough to be polluted by email ? Yes => Maybe register to a mailing list. No => That website lost a user"

25-05-2017 15:47 djs55 I don't think I'm the only guilty one, but I don't know who the others are :)

25-05-2017 15:47 avsm sounds sensible drup

25-05-2017 15:47 avsm i think djs55 me and amir

25-05-2017 15:48 avsm i have no idea where my password is

25-05-2017 15:48 avsm this answers the mystery of the missing emails

25-05-2017 15:48 yomimono so possibly amir is our only hope

25-05-2017 15:48 avsm or lars

25-05-2017 15:48 avsm lets take this discussion offline :P

25-05-2017 15:48 yomimono good call

25-05-2017 15:48 avsm (but not the mailing list, as it'll get lost)

25-05-2017 15:48 yomimono talex5 wanted to tell us about the amazing progress he's making with capnp

25-05-2017 15:48 yomimono (if I'm reading this right)

25-05-2017 15:49 talex5 Hmm, not really. It's quite a way for being ready for use.

25-05-2017 15:49 talex5 *from

25-05-2017 15:49 avsm i added that as https://github.com/pelzlpj/capnp-ocaml/issues/12 -- looks like we will migrate it to the mirage/ org

25-05-2017 15:49 talex5 A rough skeleton is at https://github.com/mirage/capnp-rpc

25-05-2017 15:50 talex5 Mostly I've been working on the schema compiler that avsm linked above first.

25-05-2017 15:50 talex5 (including a jbuilder port, but that loses some runtime performance currently)

25-05-2017 15:50 avsm waaat

25-05-2017 15:50 avsm how does the build system affect runtime performance?

25-05-2017 15:51 talex5 Module aliases and flambda don't play well together currently.

25-05-2017 15:51 avsm oh

25-05-2017 15:51 avsm is there an upstream ocaml issue about that? lpw25 and mshinwell are actively hacking on flambda, and they use module aliases a lot

25-05-2017 15:52 talex5 https://caml.inria.fr/mantis/view.php?id=7538

25-05-2017 15:52 avsm thanks!

25-05-2017 15:53 avsm phew, my assertions about lpw25 and mshinwell are backed up by their comments on that report

25-05-2017 15:54 avsm thats it from me on that

25-05-2017 15:55 avsm any other business i guess? with a whole 5 minutes left. lots going on :-)

25-05-2017 15:55 yomimono_ (sorry, ctrl-R disconnects in this IRC client apparently)

25-05-2017 15:55 avsm i have the POWER machines now from IBM, not had a chance to configure them yet

25-05-2017 15:55 yomimono_ cool!

25-05-2017 15:56 yomimono_ is there a plan for access to the machines?

25-05-2017 15:56 yomimono_ in case remote folks want to play with them... she asked, definitely not self-interestedly

25-05-2017 15:56 avsm and docs.mirage.io is rebuilding again. lots of failing libraries in https://github.com/mirage/mirage/blob/master/Dockerfile.doc#L139 if anyone cares to fix some broken libraries

25-05-2017 15:56 avsm yomimono_: working on it, suggestions for ssh key sharing mechanisms welcome :)

25-05-2017 15:57 yomimono_ avsm: tilde.town and other retro things seem to work well on just "give us your nick and your public key and say you agree to this thing"

25-05-2017 15:58 avsm nice! not seen that, will give it a try

25-05-2017 15:59 avsm thanks everyone! i need to run now. port all the libs to jbuilder, or help review :-)

25-05-2017 15:59 yomimono_ test all the things :)