Contributed by n/a on from the all-the-world-is-a-package dept.
espie@) posted a summary of the state of OpenBSD packages in a message to the tech mailing list with the subject pkg_*: the road forward.
Marc writes,
Subject: pkg_*: the road forward From: Marc Espie <marc.espie.openbsd () gmail ! com> Date: 2023-07-10 19:04:04 I spent some time during the last hackathon, talking to various people over where we're going.
Read on below the fold or take in the whole thing here (including any followups) -
I'm a bit afraid that people are going to see the ports/package framework as "not invited" (because some of the subsystems of OpenBSD are not exactly outside friendly) Right now, I got one student directly working with me on the tools, and I must say it is a good thing: I have waaaay too many idiosyncrasies. And I'm trying to get better at documenting ?` So where are we going ? there are lots of pieces missing, some of them trivial, some really annoying. - the level of testing is abysmal. I would really like for someone to have fun with regress and make it manageable. Right now, it's a pain ! More tooling to make pkg_add "fully" regression testable would be very helpful. - likewise register-plist needs some tests. I have made some progress in options to make that happen. - I would love to get some Devel::Cover stats... zero progress about this during this week (well, the main progress was that it does not work) - I converted ALL my code to perl 5.36. Having signatures is a HUGE change, not wrt actual code, but with readability. It makes perl code look like "other" programming languages, in a good way, thus making it sooo much easier for new people to get in. As for the actual code changes: - we need more code to deal with "global state", mainly /etc/rc changes and messages. Those files are going to move around, which means it's going to be GLOBAL state, because it can move from package to package. I would really really love to have regress test cases first. - we still need better Vstat code that deals with symlinks. There's one test that actually fails... and writing more tests should be simpler. - the code dealing with "old libs" is still incredibly ad-hock and needing some love. - there's something fishy in pkg_create wrt dependency handling... figure it out, AND make it work with tags as well. - fragments building is oversimplified, that's the main thing holding back ocaml "smart fragments". - texlive wants some love. That one is pretty much mine: a bit of targets in bsd.port.mk, a bit of "hints" code in update-plist, and also the ability to say "oh, don't package this, it's already in". Nothing really difficult, it's just that texlive is large... - displays and codes from pkg_add/pkg_info. The main culprit right now is FETCH_PACKAGES, it could be better. - "libsets". Updating wantlibs is really a pain for big changes. I have a keyword called libset to deal with that. It requires changes to bsd.port.mk and to pkg_add. Simply put: a libset would have a version and say "we need those libs to work". If a libset got bumped, then we wouldn't say the affected libs are a problem if they change ! This is mostly a question of writing the code. - lib-depends-check: the script AND its dependent libraries need a lot of clean-up, so that an average person can understand them. Then we can deal (probably) with subdirectories (which we don't) - dpb changes: the "files needed" change. I have all the pieces to figure out which files and directories a given build requires: in the ports tree, MAKEFILES_LIST + PKGDIR + PATCHDIR + FILESDIR + package + \ distfiles would be enough. I need to actually provide this. This would enable TWO huge things: -> disconnected builds, with a cluster that doesn't use nfs, but runs openrsync \ instead -> separate builds into separate chroots (roughly 6000+ links per build) - regress tests: should be much easier now that diskspace is (mostly) not an issue - roaching while fetching: the main part would be to figure out which part of roach \ we actually need to plug in after fetch. Having a sqlite database is actually a \ huge hindrance. dpb could output "decent" logs, and it should be easy to coerce them. I probably missed some entries, but this is what occupies my ports/pkg* thoughts. Help welcome. Not perl solutions NOT welcome.
As Marc says here, help is welcome, and there are plenty of things to dive into for the qualified and/or adventurous.
So if you feel that you belong in one or more of those categories, please dive in and check what you can do!
(Comments are closed)
By Marc Espie (espie) espie@nerim.net on
To complement my own email:
in recent years, I've tried real hard to NOT be that guy who writes his tools in the cellar, and manages to push away everyone who wants to help.
I will readily admit that my code is somewhat complicated, and I understand that not everyone will like perl.
That's one reason why the move to 5.36 function signatures was so important to me, even though it brings exactly ZERO new functionality to the tools.
Over the recent years, I have forayed into various other areas of OpenBSD development, and I will readily admit: we're not the most welcoming bunch of developers. Now, there's one good point to it: we will hate you in person. You don't have to jump through hoops and get through a convoluted set of admission procedures to submit an issue.
Nooo, you just submit to the mailing-lists, post some issue, and get yelled at.
Maybe we should market "GIAAS" (Getting Insulted as a Service) because we are very good at that.
So... I don't want to be that guy. I won't name people, but I don't think we're the best project when it comes to enlisting new guys and helping them get their feet (the ports tree bunch is somewhat better than the rest in my opinion)
Replying to some well-meaning guy about the state of so and so "just wait, it's going to happen" is NOT something that scales. And frankly, it's not the image I want to give.
So, yeah, making things easier to understand, and welcoming other visions, and heck, managing to get other people to do part of the work for me is what it's all about.
I acknowledge that PR and talking to would-be developers is an activity that takes a lot of time (and if you're a noob, please do your homework as best as you can so you don't waste our time OR don't get ignored), but we have to think about the next generation (and hope we don't get any bald guys with an affinity for earl grey, hot)
Comments
By Chris Bennett (chrisbennett) webmaster@bennettconstruction.us on bennettconstruction.us
I just spent a little over an hour looking at this, getting a handle on my negative emotions about all of this.
I don't know how emotional others feel, but I sure did receive over the years quite a few emails from others "off list" who specifically requested that I not even mention them online. Sounds a lot like cancer that ever so slowly creeps in.
>I will readily admit: we're not the most welcoming bunch of developers. Now, there's one good point to it: we will hate you in person. You don't have to jump through hoops and get through a convoluted set of admission procedures to submit an issue.
>Nooo, you just submit to the mailing-lists, post some issue, and get yelled at.
>Maybe we should market "GIAAS" (Getting Insulted as a Service) because we are very good at that.
Extremely good. So good that at least 3/4 of my time dealing with emotions about things that have been said to me on every mailing list was related to that. I don't know if I really want to explain the last 1/4. I cannot unsay that, but my father also suffered indirectly.
>So... I don't want to be that guy. I won't name people,
Well, why the hell not? I keep seeing "I won't name people" over and over. All of the unimportant people get told they are stupid, lazy, didn't RTFM, should go away and never come back, etc.
Is the social dynamic at the developer level so fragile that another developer can't be told publicly on a mailing list reply that they are being an unhelpful jerk and should improve their behavior or just not reply?
I doubt that there is any single action that could be taken that would help out new people more than seeing that. I mean recent new people, like maybe around a year or two.
>but I don't think we're the best project when it comes to enlisting new guys and helping them get their feet (the ports tree bunch is somewhat better than the rest in my opinion)
IMO, not very good at all. The opening window for new people is very short. Too short.
Ports is better in a sense, but it seems to be made up of a small number of helpful people who are really good.
They are very helpful, but where are the people who could be helpful who aren't porter super heroes?
I also have noticed that it seems like the expectations that new people should know increasingly more technical details about OpenBSD in every sense seems to be climbing. Perhaps that is a reflection of the fact that existing developers are getting better and better at understanding the innards of OpenBSD? I don't know, but I think it is a reasonable guess.
Finally, I think that "the developers are too busy" has become a bit of an excuse not to participate in the group as a whole.
OpenBSD is working pretty damn good right now. Sure, new devices and security are very important. Always!
But does project X really have to make it to this next release? Could it wait until the release after that?
Meanwhile, maybe that or those developers could put in a few weeks of work helping new people come in? Answering their questions, pointing out problems, telling them exactly what code, manuals and outside material to read in particular. Maybe explaining some obtuse problem that exists because of some hack that was done to fix some problem, etc.