p2k23 Hackathon Report: Jeremy Evans (jeremy@) on Ruby ports cleanup, database progress, and more

Next up in our reports from the p2k23 hackathon is one from Jeremy Evans (jeremy@). Jeremy writes:

My travel to Dublin started off not so great, with the airline figuring out they had to replace the copilot's chair in the cockpit after everyone had boarded, forcing everyone to deplane and then reboard an hour later. I ended up getting to Dublin a couple hours later than scheduled. This was the day before the hackathon started, so thankfully I didn't miss any hacking time. After I arrived, I took a brief nap, then found out where the hackroom was.

I headed over to the hackroom, then helped stsp@, kn@, and Tom setup the hackroom. We started out being short on tables, but thanks to the hotel, we got to the point where most developers had their own table. This hackathon was the largest ever in terms of developers for a ports hackathon, and the room was just large enough to accommodate everyone comfortably.

I started out p2k23 going through all of our Ruby ports, and either updating them if I thought they were still relevant, or adding them to the list of ports I thought should be removed. We have a lot of Ruby ports, and that process took over a day. After completing that, I sent out the list of old and obsolete Ruby ports and asked for approval to delete them. There were a total of 72 separate ports.

After that, I worked on porting MySQL 8.1.0 to OpenBSD. OpenBSD removed its MySQL port back in 2014, after replacing it with MariaDB. However, in the 9 years since, MySQL and MariaDB have continued to diverge, to the point that software that works with one does not necessarily work with the other. I have software that I would like to test against both, so I thought it would be a good idea to port MySQL. After a number of hours building and fixing minor issues, I eventually got stuck with the timer implementation, as OpenBSD does not support either timer_create or EVFILT_USER.

Not knowing how to fix that, I switched to trying to port Babelfish for PostgreSQL, which is a PostgreSQL fork and extension that makes PostgreSQL listen on the Microsoft SQL Server port and support T-SQL syntax on it. This was quite a pain to port, as it requires 3 separate distfiles that interoperate, and it expects that you install some parts to their final locations before you install the rest, which is complicated to do in the ports fake framework. In hindsight, it may have been better to split the parts into separate ports. I did eventually port it, but the port was kind of a mess, and from my testing, the Microsoft SQL Server emulation was missing quite a few things. I sent out the created port the start of the third day of the hackathon, but due to the issues, did not propose it for commit.

After that, I went back to the MySQL 8.1.0 port. I analyzed how EVFILT_USER was used in the timer implementation, and found a way to modify the timer implementation to not require it. Then I continued to port it. It took quite a long time to build on the old laptop I was using, occassionally running into problems that required patching. I ended up finishing the port on the fourth day of the hackathon, and it passed all of MySQL tests I had. I sent the port out for feedback, but nobody so far has indicated that they would like this in ports. I may maintain it outside of ports.

In addition to the old laptop I was using, I had brought with me a newer laptop that could not boot OpenBSD. After discussions with bluhm@, mlarkin@, and robert@, it turns out that robert@ had a similar laptop and had a simple kernel diff that allowed OpenBSD to boot. As I don't typically compile custom kernels, it took me a while to get the diff working, mostly because I was not familiar with how to use rdsetroot to set the ramdisk filesystem in the custom bsd.rd I compiled. I eventually got that figured out and was able to install OpenBSD on the new laptop, and copy over my files from the old laptop. That made later work on the hackathon significantly faster.

During the hackthon, I discussed with espie@ about the complexity of ruby.port.mk, which is the most complex port module in the OpenBSD ports system. I looked for ways to decrease the complexity, and found that one of the best ways to reduce it would be to remove the support for building JRuby ports. We hadn't used it to build JRuby ports since 2016, so there didn't seem a need to keep it. When testing the quirk to remove it, I found that the support had been broken since the update to JRuby 9.3 back in 2021. I made a few additional changes, to remove a couple of other features for running tests that no longer work with modern Ruby. I committed that change on the fifth day of the hackathon.

Next I worked on an update to audio/mac (Monkey's Audio Codec), as the port was many years old, and based on an old Linux port. In the last decade, upstream added Linux support, so we could switch to using upstream's distfile. The build system changed completely, from autoconf to cmake, but after figuring it out, it built without problems. The only port that uses Monkey's Audio is Aqualung, a port I maintain (both in OpenBSD and upstream), so I tried to get Aqualung working after the upgrade. A user had already supplied a diff to allow Aqualung to compile with the new Monkey's Audio, but it didn't work at runtime. After compiling debug versions of Aqualung and Monkey's Audio, I eventually determined that was due to a endianness issue in the UTF-16 encoding that Monkey's Audio was using for the filename. I fixed that in Aqualung upstream, then backported the commits to the OpenBSD port.

The sixth day of the hackathon was my last day, as I was leaving early on the final day. I started out removing all of the Ruby ports that were OKed for removal, which took a couple hours. After that, I took another look at ruby.port.mk and looked for additional ways to simplify it. I was eventually able to remove 6 separate variables used in the port, and marked a number of variables as internal to the port. I also consolidated the SUBST_VARS and UPDATE_PLIST_ARGS setting, which was previously in 3 separate places. I was not able to finish testing the changes, as it requires rebuilding all Ruby ports. I should finish my testing in a few days, and assuming no problems, I'll commit afterward. I then worked on updating the documentation for the Ruby module. schwarze@ provided some good feedback on the documentation update, and I plan to incorporate his feedback in a future update.

At the final dinner at the hackathon, job@ and I discussed his effort to get ed22519 support into ssh many years ago. This encouraged me to work on a port for the Ruby ed22519 library, which was the last thing I worked on during the hackathon. tb@ approved the port on the condition that I test it with the RFC 8032 test vectors, so I worked on tests for that while at the Dublin airport waiting for my flight, and sent those to tb@. tb@ approved the import, and I committed it in Washington DC, while waiting for my flight back to Sacramento (which was delayed 3 hours due to weather).

I had a lot of fun at p2k23 and accomplished pretty much everything I wanted to. Thank you so much to the OpenBSD Foundation, NCSC Ireland, Tom Smyth, and all of the developers who attended.

Thanks for the report and all the good work, Jeremy!

All of this makes us even more eager to see the next release.