OpenBSD Journal
Home : : Add Story : : Archives : : About : : Create Account : : Login :
g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more
Contributed by rueda on Sun Sep 11 00:00:12 2016 (GMT)
from the bin-bin-bin-patches! dept.

Here's yet another g2k16 hackathon report, this one from Antoine Jacoutot, who writes:

This was my first time in Cambridge and I must say I really enjoyed the place. It's a gorgeous town and avsm@ and Gemma's organization of the hackathon was just perfect. So first of all, thank you very much to them and to the OpenBSD Foundation.

The one thing I wanted to work on or rather talk about during that week was having a way to provide binary updates (both for the base system and packages) to ease maintenance of OpenBSD fleets. While it's perfectly possible to build your own stable releases and packages, it's a work that could be done centrally instead of having each and every user come up with his own solution. Plus that would mean we could get official signed updates.

To be honest, I was expecting some reluctance since it would mean a change in our workflow but Theo has been very opened to the idea as long as it's done properly and is up to OpenBSD standards.

So robert@ and I started hacking on it, we kind of knew how we wanted things to be done since we've been doing a similar thing in M:Tier for years outside of the official tree along with jasper@. Robert worked on the build infrastructure (to create the actual binary patches) and my chore was to write the utility that would fetch, verify, install and eventually rollback these patches.

Binary patching is a complex subject so we wanted things to be baby simple. Actually "binary patching" is not what we will really do here (i.e. it will not be a binary diffing/patching utility).

The first bits are in and this is how we would like things to work eventually:

  • fetch the tarballs containing the updated binaries/kernels/files
  • verify them with signify
  • create a rollback tarball containing the installed files that are about to be replaced
  • extract the syspatch tarball to a temporary directory
  • safely install(1) the extracted files on the system
  • if something goes wrong or if a patch introduces a regression, rollback using the tarball we've created above

That's pretty much all. We are not going to implement things like rollbacking a particular patch (but only the last installed one) because the goal is to be synced against stable and not running a Frankenstein system (patches are cumulative and some may depend on others).

Once we have syspatch in place, we can start looking at offering binary packages updates for stable (all the build infrastructure is of course already available, so that's just a matter of integrating it in our stable update workflow).

Besides that, I've worked on a few corner cases that our rc.d(8) does not cope well with. That's not committed yet because it requires a bit more work and beck@ gave me a couple of other ideas that I'd like to implement at the same time. So stay tune for upcoming changes...

Of course, I took a ride under ports/ for a while, adding some USE_WXNEEDED bits here and there (unbreaking gnome-shell and gdm) and jasper@ offered me the huge honor to remove devel/glib (i.e. glib-1.X!) from the tree. I updated a few things as well and sync the login_krb5 port to match some recent changes in base.

One night, fill with boredom^beer, jasper@ and I went to portroach.openbsd.org (which lists outdated ports in the tree) and started an update rampage

All in all it was again an amazing week amongst amazing people!

Thanks for the report, Antoine!

[topicopenbsd]

<< g2k16 Hackathon Report: Vincent Gross on iked(8), armv7 and sys/netinet[6] | Reply | Flattened | Collapsed | DragonFly BSD adopts uses of LibreSSL >>

Threshold: Help

Related Links
more by rueda


  Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod 18/84)
by Anonymous Coward (77.166.151.40) on Sun Sep 11 08:05:56 2016 (GMT)
  Binary patches are coming!
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod 5/85)
by Renaud Allard (renaud) (renaud@allard.it) on Sun Sep 11 08:25:10 2016 (GMT)
  Wow, official binary patches is something I thought would never happen.
That's an amazing news. I really want to thank you with this.
Let's go donate a little bit more :)
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

       
Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod 3/75)
by Anonymous Coward (217.84.137.20) on Sun Sep 11 21:30:55 2016 (GMT)
  > Wow, official binary patches is something I thought would never happen.
> That's an amazing news. I really want to thank you with this.
> Let's go donate a little bit more :)

Indeed. I think there were even fairly recent discussions on misc@ where the consensus was that the team just didn't have the manpower to take care of it.
This would be an awesome addition, especially for people with not so powerfull machines.

  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

         
Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod 1/75)
by Anonymous Coward (51.255.202.66) on Sun Sep 11 23:46:08 2016 (GMT)
  > Indeed. I think there were even fairly recent discussions on misc@ where the consensus was that the team just didn't have the manpower to take care of it.

That was before they decided to scrap future CD-ROM releases. I guess the time that has been freed up as a result of not having to do anymore CD-ROM releases is now being put to good use.

> This would be an awesome addition, especially for people with not so powerfull machines.

Exactly. Especially those with little disk space. Having said that though, as the FAQ suggests, compiling a system puts a lot of stress on the machine, so is a good way to figure out how good the system is. That kind of test is not suited for all machines though.
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod 8/92)
by journeysquid (Tor) on Sun Sep 11 15:26:34 2016 (GMT)
http://www.openbsd.org/donations.html
  Good work! Hopefully syspatch won't have the same shortcoming that MTier's openup does where it blindly installs patches for the X sets when they are not installed.

Does syspatch check existing files against checksums before replacing them?
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

       
Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod 4/74)
by Anonymous Coward (37.220.35.202) on Sun Sep 11 23:49:01 2016 (GMT)
  > Good work! Hopefully syspatch won't have the same shortcoming that MTier's openup does where it blindly installs patches for the X sets when they are not installed.
>
> Does syspatch check existing files against checksums before replacing them?
>

I would like to think they'd consider moving to using a C program instead of a shell script if they want to start introducing that kind of complexity. For one thing, a C program can use pledge(2).
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod 13/91)
by Gerardo Santana (70.113.112.97) (gerardo.santana@gmail.com) on Mon Sep 12 05:40:47 2016 (GMT)
https://github.com/santana
  Congratulations to Antoine for this accomplishment.

As the original author of the binpatch subsystem I remember struggling with convincing the team to publish the errata in a more easy to consume format by a script.

Antoine, can you push forward an initiative to do that? That would allow anyone to automate binpatch building even further.
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

       
Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod -11/75)
by Anonymous Cowboy (213.252.247.231) on Mon Sep 12 06:41:01 2016 (GMT)
  > Congratulations to Antoine for this accomplishment.
>
> As the original author of the binpatch subsystem I remember struggling with convincing the team to publish the errata in a more easy to consume format by a script.
>
> Antoine, can you push forward an initiative to do that? That would allow anyone to automate binpatch building even further.

I would be interested to know what kind of format did you have in mind?
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

         
Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod -2/74)
by Gerardo Santana (70.113.112.97) (gerardo.santana@gmail.com) on Mon Sep 12 11:38:05 2016 (GMT)
https://github.com/santana
  > > Congratulations to Antoine for this accomplishment.
> >
> > As the original author of the binpatch subsystem I remember struggling with convincing the team to publish the errata in a more easy to consume format by a script.
> >
> > Antoine, can you push forward an initiative to do that? That would allow anyone to automate binpatch building even further.
>
> I would be interested to know what kind of format did you have in mind?

Back then I proposed one based on XML. But anything with structure is ok. What matters is to clearly mark up each part of the errata.

From there we can automatically generate errata.html as well as the Makefile targets.
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod 22/36)
by jine jack (kasstri) (fgffcvgffhff@gmail.com) on Thu Oct 20 20:55:08 2016 (GMT)
http://www.clavier-arabe-pro.com/arabic-keyboard.php
  Indeed. I think there were even fairly recent discussions on misc@ where the consensus was that the team just didn't have the manpower to take care of it.

That was before they decided to scrap future CD-ROM releases. I guess the time that has been freed up as a result of not having to do anymore CD-ROM releases is now being put to good use.
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod 0/0)
by Kenna Kennedi (182.182.122.65) (personyoga@gmail.com) on Wed Dec 21 07:35:35 2016 (GMT)
  Thanks for telling me about the g2k16 Hackathon Report. I really hope that the whole event would've had gone really great. I've heard about the bestessays scam and also how it was alleged but it was all a lie. Thats a fact because I've been trusting them with my work for a long time.
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  No Subject Given (mod 0/0)
by Atwood (104.236.124.172) (zgtsqylc@eelmail.com) on Wed Dec 28 13:49:38 2016 (GMT)
Aubrey
  Thanks to ones own modern content articles, As i already have the ideal article product after only a obtain. Well before I ran across essay-on-time, Make possessed trouble selecting a decent article product. Once As i implemented you As i is allowed to remain disenchanted. At present, My group is set! forskolin extract
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  No Subject Given (mod 0/0)
by Harper (104.236.53.155) (zgtsqylc@eelmail.com) on Wed Dec 28 15:10:49 2016 (GMT)
Manning
  Checking out just about any web site may make people only desire to get a greater number of. I did an excellent i was first attractive written content with. It more than likely executed to the tier even now itís extraordinary. My spouse and i generally dispatch anyone. e cigarette starter kit
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  No Subject Given (mod 0/0)
by Davidq (45.116.232.40) (zydgqzay@imgof.com) on Wed Jan 4 13:51:49 2017 (GMT)
  Quickly this site will indisputably be famous among all blogging people, because of its fastidious articles or reviews. pure forskolin reviews
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  No Subject Given (mod 0/0)
by Davidq (182.182.122.227) (zydgqzay@imgof.com) on Wed Jan 25 10:24:28 2017 (GMT)
  There are several dissertation online websites online live pick-up not surprisingly acknowledged inside internet sites. Seaside Residences
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Chaise lounge (mod 0/0)
by Macheal (178.62.106.169) (adada@asda.com) on Mon Jan 30 09:22:32 2017 (GMT)
  You can make your outdoor sitting area even more relaxing and inviting with chaise lounges. They can be the ideal accessory to be placed near the poolside, beach or lake. Chaise lounges give your outdoor place, a whole new classy look and provide maximum comfort. After an exhausting day at work, you can relax yourself in the most comfortable manner by laying down on a chaise lounge on your poolside and reading your favorite book or listening to music. If you are looking to get the most comfortable and durable cushions for chaise lounges, Cushion Con-nection is the right place for you. Choose from a host of styles and designs and enhance the look of your outdoor chaise lounge.
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  No Subject Given (mod 0/0)
by Kally (128.199.165.82) (asdad@ada.com) on Sat Feb 4 09:56:10 2017 (GMT)
  I'd like to see one's put. It really is more advanced than discover most people verbalize while in the core in conjunction with readability on this subject area crucial space are likely to be effortlessly observed. flyers printing mississauga
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: g2k16 Hackathon Report: Antoine Jacoutot on binary patches, ports progress and more (mod 0/0)
by mxffiles (218.11.246.179) on Tue Feb 7 07:21:53 2017 (GMT)
  This is a very good post which I really enjoy reading. It is not every day that I have the possibility to see something like this. Software mxf Software mxf converter free download to convert HD camcorder files. ts converter convert ts video files to avi, mp4, wmv, mov mts to avi mp4 mov mkv iMovie, FCP/FCE with mts converter, so to convert mts files for your PC and mobiles. mod converter and convert tod files just free download mod video converter. m2ts
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

[ Home | Add Story | Archives | Polls | About ]

Copyright © 2004-2008 Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to April 2nd 2004 as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. Some icons from slashdot.org used with permission from Kathleen. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. Search engine is ht://Dig. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]