Irc discussions from #mirage on 01-02-2017

Written by avsm
Classified under: irclogs
Published: 2017-02-01 (last updated: 2017-02-01)
<yomimono> hello!
[16:02] <yomimono> this is the time at which I said we'd have our biweekly (fortnightly) catchup
[16:02] <yomimono> so let's do that!
[16:02] <yomimono> The agenda is at https://github.com/mirage/mirage-www/wiki/Call-Agenda .
[16:02] <hannes> o/
[16:02] <yomimono> I don't see our friendly camel logger, so we'll depend on _whitelogger for logs for the catchup.
[16:02] <avsm> Sounds good! I am currently compiling up the self-hosted infra on a brand new packet.net machine :-)
[16:03] <djwillia> hi all
[16:03] <avsm> I will paste the logs into canopy-data
[16:03] <avsm> hi djwillia!
[16:03] <yomimono> thanks avsm! :D
[16:03] <yomimono> Last meeting's call for a summarizer is still open -- if you're here and you want to write up a summary of what we've discussed and decided, that would be very welcome!
[16:04] <yomimono> If you do so, let everyone know on mirageos-devel.
[16:04] <avsm> I'll send a complementary camel bubblegum sweet to any volunteers
[16:04] <yomimono> :D
[16:04] <avsm> they are delicious
[16:04] <amirmc> Even a few short paragraphs would be helpful. No need for it to be extensive prose :)
[16:05] <mort___1> always wondered why camels are always chewing. now i know.
[16:05] <yomimono> Seeing nothing that's been added before the first agenda item...
[16:05] <yomimono> the first thing we have is "MirageOS 3 beta - it's so great and extant!"
[16:05] <hannes> (I have evidence that camels are delicious, indeed)
[16:06] <yomimono> noddy and hannes sent PRs for the last bits we were missing yesterday, so now we have a complete-ish repo at https://github.com/mirage/mirageos-3-beta
[16:06] <yomimono> instructions on trying it out are in the README
[16:07] <yomimono> several user libraries (irmin, nbd, etc) are not yet accounted for there, but the libraries that get used by code generated by `mirage configure` should be all good to go.
[16:07] <yomimono> Please try it out!
[16:07] <mato> Should those instructions also recommend to remove any mirage-dev remote?
[16:07] <avsm> I've been following it and working well so far. Only minor fail was I used a 4.02.3 installation and got a slightly cryptic failure
[16:07] <yomimono> Good catch - yes, mato, they should, since the versions there will be above the actual release versions referenced in the beta repo
[16:07] <avsm> Should we put a hard 4.03.0 restriction on mirage.3.0.0 so that attempting to use the CLI from OCaml 4.02.3 will fail?
[16:08] <reynir> o/
[16:08] <mort___1> 58
[16:08] <mort___1> (sww)
[16:08] <avsm> There are also presently bad checksums for nocrypto in the staging tree so that will fail
[16:09] <yomimono> avsm: that should already be in place, if not yes
[16:09] <thomasga> (I would love to be able to continue to use my libraries on 4.02.3 if possible)
[16:10] <yomimono> thomasga: restricting the mirage frontend to 4.03.0 and above shouldn't stop you from doing that, though
[16:10] <thomasga> so I'm fine with the restriction then :p
[16:10] <yomimono> yay :D
[16:10] <amirmc> So with all those PRs now merged, are we officially in beta mode?
[16:10] <yomimono> yes, once the checksum for nocrypto is fixed I'll send a mail to the list
[16:10] <amirmc> Fabulous!
[16:11] <avsm> amirmc: codewise yes, I am still working hard on the new website and hope to have something to show around more widely by Friday
[16:11] <avsm> (it's a fairly simple Jekyll-based effort, so the main change is that deployment is far simpler, via the stock static_website mirage-skeleton example)
[16:12] <yomimono> there's a lot of additional stuff we need to do - infrastructure and docs namely - which are tracked at https://github.com/mirage/mirage/issues/592
[16:12] <yomimono> (from memory!)
[16:12] <avsm> that's a fairly ridiculous number of new packages. Great effort tagging and getting all those staged!
[16:12] <thomasga> I concur, this is an epic release :-)
[16:12] <avsm> One thing to test is upgrades from mirage2 to mirage3 -- in particular, root packages installed in the old world may render mirage3 not installed without a specific version. We should possibly recommend `opam install mirage.3.0.0`
[16:12] <yomimono> huge thanks to all of you :)
[16:13] <avsm> and to you for your release management epicness, yomimono!
[16:13] <mato> ^^^ +100
[16:13] <amirmc> Most important from that list (for the beta phase) are some kind of migration docs  … but that's an agenda item so we'll get to it.
[16:13] <hannes> we'll need a new pcap-format release (since older versions had depopts on mirage-net-0.X, and now are conflicting with the new mirage-net packages) -- I prepared https://github.com/mirage/ocaml-pcap/pull/23 in case someone has strong feelings about the print and mirage subdirs, please say so
[16:14] <mort___1> hannes: thanks, looking at that, will get to it this afternoon
[16:14] <mort___1> i doubt i have strong feelings though :)
[16:14] <avsm> and thanks for the DNS PR hannes. I'm not sure it's a good idea to migrate ocamlfind package names in the same release as a topkg migration, but I guess we can fix up upper bounds in one go.
[16:16] <hannes> avsm: I confused myself within 4 hours several times over dns-lwt (the lwt-unix stuff) and dns-lwt-core (the actual core, independent of unix).. and there are few reverse dependencies, thus I decided it is worth doing
[16:16] <avsm> hannes: yeah might as well be consistent
[16:17] <thomasga> (please keep consistent names — e.g. same names for packages and libraries :p)
[16:18] <yomimono> any other TODOs or questions?
[16:19] <avsm> I'm good -- have progressed through to figuring out how to configure static IP and bridging for the infrastructure
[16:19] <hannes> will there be a release party?
[16:19] <avsm> (got past nocrypto with a pin)
[16:19] == noddy [~self@anon-63-116.vpn.ipredator.se] has joined #mirage
[16:19] <yomimono> hannes: I'll be holding a party in Madison WI, all are welcome to attend :P
[16:19] <avsm> hannes: yes, we must have one! perhaps morocco is the right place :-)
[16:20] <amirmc> Shall we move on to the next item?  Or did we cover that already? "Revisions to introductory materials (@yomimono with thanks to @mor1)"
[16:21] <yomimono> not yet! I just wanted to mention that I'm working on tutorial rewrites as discussed by me, drup, and mor1
[16:21] <yomimono> I'm also carving the mirage-skeleton repository into tutorial stuff, device usage examples, and higher-level applications
[16:21] <mato> By the way, there's still the rather annoying user-visible issue of "mirage --help" and "mirage <command> --help" dying with a cryptic error. hannes, I believe you had some fixes for that?
[16:21] <yomimono> I've removed a bunch of stuff that I thought was redundant -- you can see my branch at https://github.com/mirage/mirage-skeleton, branch split-examples-from-tutorial
[16:21] <mort___1> yomimono: as i'm clearly a bit rubbish at keeping up — point me at the next bit it would be useful to re/write, and i will
[16:22] <yomimono> mort___1: awesome, i'm merging your prose changes in right now :D
[16:22] <avsm> yomimono: mort___1: I've created (but not yet documented) a new skeleton website at https://github.com/mirage/mirage-bushel-www. It's easy to port mirage-www changes over to it, but we should sync up on thursday evening or so I expect
[16:22] <yomimono> well, not *right* now, right now I'm dinking around in IRC
[16:22] <mort___1> @mato: +1 was typing that exact same comment just now :)
[16:22] <hannes> mato: https://github.com/mirage/functoria/pull/101 fixes this and removes -f.
[16:22] <avsm> the --help thing is getting me all the time. I never use `cmd help`, and support the -f removal
[16:22] <mort___1> +1
[16:23] <yomimono> I think I'm the only one who's got a :( about removing -f, so merging 101 is probably the right thing to do
[16:23] <hannes> yomimono: great (reviving tutorial, updating mirage-skeleton): there are also examples in tcp/examples and https://github.com/cfcs/mirage-examples which may be worth looking into
[16:23] <thomasga> (bouhouhou about `-f`)
[16:24] <yomimono> hannes: yes! I'd like to have a reasonable home for the things that are just "here's how you use X", and have there be more of those, in addition to the list of application examples you've said would be nice
[16:24] <mort___1> (sorry, to clarify: i don't have strong opinions on `-f` — it's a nice but not necssary — but `—help` breaking is very unacceptable)
[16:24] <thomasga> (but I see the merit of simplifying that bit of code which was quite hackish)
[16:24] <thomasga> I am fine to re-add `-f` later if life becomes too hard without it
[16:24] <avsm> if anyone gets a chance to try the new opam2 local build functionality, then `opam build` should "just work" locally on the mirage3 unikernels
[16:24] <avsm> once `mirage configure` has been run
[16:25] <thomasga> avsm: that's excelent!
[16:25] <thomasga> it's a pretty nice side-effect of both independent features :p
[16:25] <thomasga> (or maybe was it planned?)
[16:26] <avsm> https://github.com/ocaml/opam/pull/2822 is the PR upstream
[16:26] <avsm> (Louis requests feedback on it, so now's a good time)
[16:27] <yomimono> I'll retroactively add an agenda item noting that :P
[16:28] <yomimono> avsm, did you have more to say on migration docs, our next item?
[16:29] <avsm> Only that I'm documenting my migration process on the live infrastructure, and intend to serialise it into a document
[16:29] <avsm> Current progress is: 1) new jekyll-based static html website (ongoing, should be 2 more days to show off on list for wider feedback/contribs) 2) Travis/DataKit deplotyment support via static_website
[16:30] <avsm> and 3) figure out all the packet.net runes for deploying the static ip website (bridging setup etc)
[16:30] <amirmc> Thanks. That'll be useful for folks who'll be coming at this cold.  I know the folks at Ericsson will find it useful.
[16:30] <avsm> all seems fairly straightforward, will just take a couple of days to settle down.
[16:30] <avsm> amirmc: excellent, i'll put it up as soon as I have something working.
[16:31] <yomimono> other items?
[16:32] <avsm> How's the hackathon prep going?
[16:32] <amirmc> Anyone fancy writing some blog posts?  I think GemmaG might have been chatting to people. :)
[16:32] <avsm> Sounds like it's a full house!
[16:32] <hannes> there seems to be a book chapter on MirageOS in a book soon to be published by Christopher Wilson https://www.manning.com/books/the-transparent-web ...
[16:33] <yomimono> that's twopoint718 who's often here, although I don't see him atm
[16:33] <avsm> wow, can't wait to read that
[16:33] <hannes> it's like a rollercoaster... days where I get up to 5 mails with some people dropping out, others joining.. I'd guess we'll have a stable participation of more than 30 people!
[16:34] <hannes> apart from that I try to stay away from organising too much -- there is still code which needs to be written
[16:34] <thomasga> we definitely need more books on MirageOS :-)
[16:34] <yomimono> +1
[16:34] <yomimono> The Littlest Functor
[16:34] <amirmc> Maybe after this round of breaking changes has settled ;)
[16:35] <hannes> amirmc: I'd like to (at least contribute) to a post on 'how MirageOS3 got smaller and more dumb' -- taking a look at generated main.ml, and generated binaries (they are smaller, aren't they? ;)
[16:36] <thomasga> "The Unikernel which does Nothing"
[16:36] <amirmc> hannes: That sounds great!  thomasga: That sounds like one of talex5's posts :P
[16:36] <yomimono> https://somerandomidiot.com/blog/2014/07/25/doing-nothing-in-mirage/
[16:37] <amirmc> Oops! I meant yomimono. Sorry!
[16:37] <yomimono> inspiration for the `noop` example which mor1 wrote into the tutorial already!
[16:37] <mort___1> looks like a default build of the noop unierkenel in teh new skeleton comes out at 2.7MB for me
[16:37] <yomimono> so perhaps we're already a lot of the way to that book :D
[16:37] <thomasga> yea I remember! That would make a pretty cool book title :p
[16:37] <mort___1> (targeting unix on osx)
[16:37] <avsm> it may or may not be a coincidence, but the latest member of OCaml Labs is 4 hours old today, so we have multiple releases
[16:38] <yomimono> :D :D :D :D :D :D :D
[16:38] <yomimono> is it legal to hire babies in the UK?
[16:38] <mort___1> not yet
[16:38] <avsm> gotta teach them good programming practise early
[16:38] <mort___1> afaik
[16:38] <hannes> :D
[16:38] <avsm> especially garbage collection
[16:38] <yomimono> hannes: should we add that blog post to the list of things to do on #592?
[16:38] <avsm> Alright, I can report a successful solo5 and xen build with the beta branches, for mirage-skeleton. wooop :-) not live yet but as soon as I get static ipv4 addresses...
[16:39] <avsm> mort___1: would be good to strip the debug symbols and compress and see how it looks
[16:39] <hannes> yomimono: sure
[16:40] <mort___1> `strip main.native` —> 2.08MB
[16:40] <thomasga> with flamba?
[16:40] <mort___1> bzip2 -v9 —> 584kB
[16:40] <hannes> sounds pretty big too me... is the lto branch still active? ;)
[16:41] <mort___1> ocaml 4.04.0
[16:41] <mort___1> (system compiler on osx currently)
[16:41] <avsm> So we will probably have a couple of weeks to look for issues with the release, update website/etc, and then announce the release. amirmc is coordinating that with the Linux Foundation -- any ideas on dates yet?
[16:42] <hannes> LF + xen + docker? + IBM + Ericsson ?? or who's involved in the announcement?
[16:42] <avsm> hannes: yes we should be able to experiment much more easily with the lto and flambda branches now. Takayuki's performance harness is making excellent progress. It just needs solo5 support for mirage-profiling...
[16:42] <amirmc> No feedback on dates yet.  Waiting to find options that work for all parties, which seems to be rticky.
[16:42] <mort___1> yomimono: i have an aob / mirage3 item i just remembered for the agenda — shoudl i add to wiki or just bring it up at the end?
[16:43] <avsm> now's probably fine...
[16:43] <yomimono> we're out of agenda items, so go for it
[16:43] <mort___1> is a question more than anything — where does talex5 tracing/profiling stuff sit with mirage3 currently?
[16:43] <mort___1> fully supported? untested?
[16:43] <mort___1> somewhere between?
[16:44] <amirmc> hannes:  First thing to do is figure out possible dates.  LF is handling the release (though we'll write/rewrite), and Docker is also involved.
[16:44] <yomimono> talex5 got the fork building again a couple of weeks ago.  I've been building with it occasionally for unix/xen
[16:44] <yomimono> no tracing for solo5 as yet.
[16:44] <talex5> It's mostly not specific to Mirage anyway.
[16:44] <yomimono> related to that, I'd like to make a more compelling example for its use - right now the unikernel in "tracing" in mirage-skeleton is not super interesting
[16:44] <yomimono> needs more threads!
[16:44] <mort___1> ok— are there up-to-date instructions anywhere for how to use?
[16:45] <mort___1> context is: i have a masters student who's project is to use the (mirage) tracing stuff to build magpie-like tools
[16:45] <talex5> https://github.com/mirage/mirage-profile ?
[16:45] <yomimono> no change to usage instructions since some time ago, but I can make sure that the most discoverable ones are easy to find and accurate
[16:45] <mort___1> and now he's getting going i'd like him to turn to mirage3 rather than mirage2 if at all possible
[16:45] <mato> Tracing for Solo5 needs (afaik) figuring out a shm interface to the profile data. Could be done with a ukvm module. Probably not doable/worth doing for the virtio target.
[16:46] <mort___1> ok— if having a more interesting example using tracing is useful, i can point him at that (i just told him to go and build himself an example anyway)
[16:46] <yomimono> excellent!
[16:46] <mort___1> i shall bear ukvm/solo5 in mind when it comes to looking at how to span the gap bewteen unikenrel and host — @mato may come asking for help at that point :)
[16:47] <avsm> mato: takayuki could really use a ukvm tracing backend to compare xen and solo5, vs unix.
[16:47] <djwillia> also ricardo and I would be happy to spend time on a ukvm module to assist with tracing if we can better learn how it works
[16:47] <mort___1> finally — is there a list of libs anywhere that utilise tracing particualrly?
[16:47] <avsm> mort___1: good question -- only from the dependency graph (they would depend on mirage-profile)
[16:48] <yomimono> tcpip has a lot of labeled stuff in it
[16:48] <mort___1> cool, i was hoping you'd say that :)
[16:48] <mort___1> what about cohttp (or http/af??)
[16:48] <avsm> nothing in there right now
[16:48] <thomasga> takayuki has a lot of tracing graphs
[16:48] <mort___1> (or taka's perf harness maybe?)
[16:48] <mort___1> yes, maybe i should point him at those tools first
[16:49] <mort___1> thanks all
[16:49] <yomimono> thanks mort! look forward to seeing what your student puts together :D
[16:49] <talex5> mirage-logs also forwards Logs debug to the trace buffer, so anything using that will produce some annotations.
[16:49] <mort___1> me too … ;)
[16:49] <mort___1> ah! thanks @talex5 that's useful to know
[16:49] <yomimono> feel free to direct them my way for whatever you think I can be helpful with :)
[16:50] <avsm> talex5: ah very good point. Cohttp should definitely use more Logs