Contributed by rueda on from the again-and-again-and dept.
Version 0.99 of Game of Trees has been released (and the port updated).
* got 0.99; 2024-05-05 see git repository history for per-change authorship information - make 'got fetch' work with URLs which refer to $HOME via a tilde: ~user - replace strftime %G-%m-%d with %F to prevent 2024-12-30 -> 2025-12-30 - fix spurious errors from got-fetch-http when server has no more data to send - prevent gotd notification process from exiting due to EPIPE - fix I/O hangs with TLS in got-notify-http - document http and https protocol support in got.conf(5), too - fix an fd leak in gotd's notify process causing endless CPU spin - back out got stage -R option addition; deemed too inconvenient in practice - fix got-fetch-http GET request URL; add leading slash and avoid double slashes - allow custom GOT_TEST_HTTP_PORT when running regression tests - gotwebd: add magic ".git" handling; try foo.git if repository foo is not found - expose authenticated gotd user account in HTTP notifications - gotd.conf(5) HTTP/JSON documentation fixes - fix endless loop upon Ctrl-D (EOF) input during got stage/unstage/revert -p - make gotd notifications work when 'git push' is used instead of 'got send' - make got stage -p behave the same way in interactive and -F modes for 'q' - fix lingering gotd processes from clients closing connections early - regress: prevent spurious failure of gotd test_clone_basic_access_denied - fix an issue where 'git fetch' would error or hang against gotd - use polling read in got_pkt_readn() to avoid endless hangs in gotsh
And we, too, are curious what the next version number will be :)
By Stefan Sperling (stsp) stsp@openbsd.org on http://stsp.name
Fortunately, the amount of available release numbers is infinite. We aren't about to run out of numbers.
Reply
Comments
By System Operator (46.10.166.16) on
The OpenBSD CVSweb server is at the moment defunct and shows errors about missing rlog on all main modules now
please see https://cvsweb.openbsd.org/ and descend into the sub-directories for the respective error reports..
Reply
By Anonymous Coward (2a01:4f8:c012:fec9::1) on
Hi,
I commited to github.com yesterday, and I'M wondering if this is a problem with got as well, my system time was wrong by a day and github displays the wrong date on the commit. Please see my latest post to
misc@ for the github info.
I'M wondering if code can be "hid" on a repo by commiting into the tree at a far enough time.
Best Regards,
-pjp
Reply
Comments
By Anonymous Coward (49.12.42.182) on
you are very weird, and didn't talk about anything that's true at the moment.
Reply
Comments
By Gnunix (87.233.232.236) on
wow, you are on something..
Reply
Comments
By Anonymous Coward (2a01:4f8:c012:fec9::1) on
You know if I wanted to be trolled like you are doing it right now, I'd just go back to reading golem.de or heise.de and participate in the comments. I left those websites fortunately because it got too trollsome.
Reply
Comments
By Anonymous Coward (2a01:4f8:c012:fec9::1) on
@blog admins - you can't delete someones words and leave the replies. You must indicate that you censored something otherwise someone new to the thread will think that the wrong person was replied to.
Just my 2 cents.
-pjp
Reply
By Stefan Sperling (stsp) stsp@openbsd.org on http://stsp.name
Timestamps recorded in commit objects are generated on the client-side, so if the local clock is off, the timestamp will be, too. There is no way around this while keeping Git clients capable of decentralized version control with no central server that needs to remain online to allow people to get work done.
However, each commit must still refer to its parent commits by a hashed identifier (either SHA1 or SH256 depending on the repository format used). This link to previous commits should take priority over timestamps when displaying commit history. The timestamp is just informative.
Reply
Comments
By Sebastian Rother (83.135.73.210) on
You could Clock-Sync the Clients (Not sytemmwide) with the Server they are commiting to. Where ist the Issue? You can do this Srver-Side If the Client ist VALIDATED and Reports ITS Timezone. Issue fixed...
Reply
Comments
By Stefan Sperling (stsp) stsp@openbsd.org on http://stsp.name
Sebastian, a major and persistent issue is that any volunteer's time available for work on this project is highly restricted. Consider why it took us 5 and a half years to get this project where it is today with its still fairly limited feature set. Please keep this in mind when requesting features or bug fixes (for gameoftrees or OpenBSD at large). Developers are already taking on as much work as they can reasonably afford to.
That said, thanks for pointing out this issue with unsynced clocks and commit timestamps, I am well aware of it.
For the time being there are (in my opinion) more important problems left to solve. But rest assured that if this ever becomes a significant problem for us then we will find a solution.
Reply
By Anonymous Coward (83.135.73.210) on
RE-Validate your Codebase... Codereviev..!!
You go too far since you want the 1.0 Release. Reconsider your Goals...
Security First...
Reply
Comments
By Sebastian Rother (83.135.73.210) on
IT was me, forgot my Name since it's a Mobile-Phone
Reply
By System Operator (46.10.166.16) on
Frog tip: the new release of Plan9 is going to be issued with the name Plan10 exactly when the Germans win WW3
and that's about as much as you need to know about how versioning really works in the real world. Victory day!
Now for some actually useful details, don't get too excited, Git isn't a protocol & it does not need adoption.
https://en.wikipedia.org/wiki/Version_control?useskin=vector#Distributed_revision_control
https://en.wikipedia.org/wiki/Distributed_version_control?useskin=vector#Distributed_vs._centralized
Nothing mandates you need or have to ever use a "distributed" or "decentralised" software tooling for OpenBSD.
There is some substantial benefit of having own tooling to access really useful protocols and services though.
This one is on the edge of being useful for a useless attempto "mandated" versioning system by popularity only
and not technical or project requirements. It is merely the zeitgeist of the post 2004 (64bit computing) hype.
Many have died on that hill, and it surely (don't call me Shirley) means it will be replaced too, quite soon..
I told you, what you need is a repository backend standardisation and conversion protocols, not trailing fail.
Or simply, normalisation tools that "work" interchangeably and reliably reducing and obsoleting this branding.
Follow your destiny repeating unlearned history every time, jump step on new gardening tools from new heights.
Reply
Comments
By Stefan Sperling (stsp) stsp@openbsd.org on http://stsp.name
The Git repository is a de-facto standard. I don't think it's the best there will ever be, but it is good enough for what I wanted to implement.
A lot of effort and consideration went into the tradeoffs of the design of this tool. If it doesn't fit your needs then just use something else or go your own way and write your own system, whatever. No reason to tell us about it.
Reply
Comments
By System Operator (46.10.166.16) on
Actually, Git is not a "de-facto" and is not likely to be a standard of any type, especially in the long term.
https://en.wikipedia.org/wiki/Git?useskin=vector#toc
https://www.google.com/search?hl=en&q=Git+standard
https://www.google.com/search?hl=en&q=Git+RFC
Face it, there is no published standard, requests for comments, and no protocol that Git provides as a server.
There is no need to justify your personal choices and preferences, these are your inalienable freedoms rights.
Nothing said and done thus far changed or solved the realistic and imperative needs and requirements of actual
repository independent software provisioning and management server systems and clients. It remains unresolved.
You're attempting outer layer replacement utilities as small conveniences and not entering the above problems.
Reply
Comments
By Stefan Sperling (stsp) stsp@openbsd.org on http://stsp.name
Well, writing RFCs for Git is not among the goals of the gameoftrees project. You'll have to look elsewhere for collaborators on a project to solve whatever realistic and imperative needs you have. I'm not sure what if anything this has to do with us.
Reply
Comments
By System Operator (46.10.166.16) on
I'll tell you, since you asked. Suppose, you convert your project repository historic metadata to Git. You are
now into the Git repository system. Your project made 25 years of commits prior to that, and after a couple of
years you discover the Git overhead is so high you must immediately revert to a really centralised system back
or your project suffers a blow of degrade and desist by churn. You have no options. You've been killed by Git.
What does this mean, well it's very simple. This really is no the complete tooling you need, and for consumers
and not providers of repository hosting services, like an independent project, suppose it's OpenBSD does need.
Reply
Comments
By Stefan Sperling (stsp) stsp@openbsd.org on http://stsp.name
Thanks for your concern.
I am well aware of scalability issues inherent in Git's initial repository design, as well as various improvements which have been made to the repository format in recent years to address these issues.
It wasn't the case until some years ago, but nowadays there are projects with larger and deeper trees than OpenBSD's tree, in a single Git repository, and they have been hitting such issues already and finding solutions for them. We can thus benefit from solutions others have already designed as repository format extensions, such as the commit-graph cache, packfile bitmaps, sparse clones, etc., and implement them as needed. The commit-graph cache being the most interesting item to add next, I think, since it would accelerate the process of enumerating commits when fetching from or sending to the server.
There are also design ideas of our own which should make it possible to add commits to the server without having to fetch everything first, such that the system works more like CVS/SVN when committing.
I had expected that such features would be needed right away for a project at the scale of OpenBSD to function. However, experience shared by people who are already using Got to work on FreeBSD ports suggests it's not strictly required just yet.
Over the last 6 years, myself and several others have been using this system for all sorts of OpenBSD-related work and haven't yet hit any problems that couldn't be addressed. In recent years Got has not been any slower or less reliable than CVS (on average -- there will always be some edge cases where one design has a significant advantage over another).
Still, version control isn't a toy, it's a core piece of infrastructure. A lot of care and careful consideration and testing is required when making big decisions about it, for sure.
Reply
Comments
By System Operator (46.10.166.16) on
Good. This sounds well, like optimism & interest in the technical details, dedication and "actual commitment".
Certainly you will have to sort these one way or another, regardless of intent and predisposition, so alright.
What about people interlocking and synchronised operation of groups of groups of people as the merge overhead.
The actual assurance everyone is working on a recent tree.. OpenBSD is an operating system, not just a tree of
drivers contributed by third party outsiders and a coordination central bunch of importers and mergers.. There
are more than just the technical actions of fast return of a slice of a cake that is just as stale as the olde
centralised system. In fact, for any size of projects from FreeBSD to Linux, Microsoft and Google, there are..
about 5 people working on the same segment and none of them concurrently. There is however a serious overhead,
in the churn of merging lax spillage of commit changes outside that micro-section. See the drivers are not OS.
The model of operation is fine for projects that are in a constant state of desync and are merge coordinators.
It is not fine for a consistent never out of sync operating system complex assurance and reliability objective
goalset. So, the proposal to convert the team of priority system integrators into mergers of defunct branches,
is really going to challenge much smaller projects that are into consistency and release engineering for real.
OpenBSD will just get hurt by Git, as it does not fit the BSD and Operating System guidelines of development..
It is fine to get it going as a userland tool to import junk that YOU do filter, adapt, convert and make work.
The conversion of the project to a decentralised system changes the project operation, and it is a big change.
So, hopefully you understand it's not the technical "prompt back" but the interoperability & coordination that
are affected by this. And nothing solves the "lock in" and "vendor capture" of the repository formats & specs.
You know it is really not a tool to solve the software coordination but cloud services migrations. It is true.
All in all, there is productivity and software engineering involved, not just time to tag and branch slicing..
Too much of the branching and slicing really does kill productivity and coordination, a problem you create for
yourselves and the big project artificially which was not really there and you did not ever need to deal with.
Let's leave it at this.. I think you know well these details and do not need the overhead of the planning now.
Reply
Comments
By System Operator (46.10.166.16) on
One more thing, you'll also need an overlay (union) file system or as it is known in BSD the null file system.
Then reliable memory file system is your next point in the navigation to efficient versioning & compilations..
We'll also throw in the reliable FUSE, even though it's not really obvious at the time, it will be needed too.
Have fun and good luck on your micro-project, to many returns of the versioning debate & well done on updates.
Reply
Comments
By System Operator (46.10.166.16) on
And did I have to remind you of the pending SMP unlocking for the main repository server which is centralised,
and that the CVS mirror setup is already distributed but not out ouf sync as the pseudo-decentralised systems..
and that file system metadata pooling also known as the gone softupdates are not replaced by journalling yet..
It's going to be challenging when the live load actually shows up in real world use after a couple of years it
will have real challenges being self-hosted on the own OpenBSD system, so plan the scenario for POSIX desyncs.
And then there is the fsync and repository application server pooled data purging and negative caching extras.
And don't forget to chroot and unveil for separation of network against metadata and data and control streams.
What about system calls classes separation and invocation location capping, have you separated model and view?
Reply
By Stefan Sperling (stsp) stsp@openbsd.org on http://stsp.name
The reality is that the Git repository format does not dictate a particular workflow. It is possible to build a centralized system on top of Git's repository format.
To see why, we only need to look beneath the user-level path@revision abstractions of the centralized version control system SVN. What we find there is a DAG. A DAG that stores versioned snapshots of trees, while storing any particular version of a particular (sub-)tree only once. Just like Git's DAG.
SVN DAG node IDs are numbers, similar to RCS revision numbers. In Git's format the DAG node IDs are hashes of content. Otherwise, the DAG models are very similar, give or take minor differences which don't matter, such as the explicit tracking of copies in SVN.
Changes committed to an SVN repository are always based on some revision number N (which identities a particular tree root node in the DAG). When a user commits their changes, this global revision number might already have stepped up, say to N+5. In this case, the SVN server rebases(!) the incoming changes onto the tree found at N+5. If no tree-wise overlap occurs the result ends up being N+6, otherwise the user has to update manually and potentially resolve merge conflicts. There is no reason why a Git repository server could not do the same.
If we look back far enough into SVN's early history, we find an interesting suggestion which came up during the initial review of design proposal in the year 2000, before any line of code was written. The idea to make SVN a distributed version control system:
The idea to use hashes of content as identifiers rather than RCS-style revision numbers also came up but was dismissed at the time.
Conversely, a distributed system can be built on top of SVN's design. Git's identifiers have the advantage that we do not need to be worried about clashes of identifiers across different repositories. But if we slice up the SVN DAG ID number space and assign non-overlapping ranges of numbers to different repositories, we can build a distributed version control system based on SVN's repository format. A similar idea was implemented in 2004 in SVK, though at a slightly higher level of abstraction.
Reply
Comments
By System Operator (46.10.166.16) on
I see your plan to use Git for the ports tree could prove utilitarian, but there is no real development there,
as it's an adaptive and ports fitting work as wrapper infrastructure rather than live application development.
Many can certainly put a lot MORE real development work extending an OpenBSD userland with applications there.
Also this "signifies" you have to find out which cryptographic functions hardware accelerators work in OpenBSD
to offload the central processor for the repository application server too, so that you have these hash tokens
generated by the server (and validated at various times as well) on top of the database and file system usage.
It is a significantly more substantial load with hardware for software offload requirements on the system, yet
we have not yet confirmed that database and file system overhead is going to be sustained on a moderately busy
repository application server, self hosted on OpenBSD with OpenBSD only infrastructure. It's a bit worrisome..
The SMP unlocking in kernel is also not SMP non-blocking file system and database application servers. Then it
adds on top hashing and cryptographic functions, and we see degrading server strength base changes in OpenBSD.
How do these align with the increased requirements of an application server combination: DB and I/O workloads?
And add the main point, what is the rationale for not fixing & extending CVS to become SVN-like operationally?
The idea to benefit from the busy repositories requirements of others improving Git as format & implementation
is alright, but does this really provide any actually solid problem solutions to the generic repository model?
Reply
Comments
By Anonymous Coward (stsp) stsp@openbsd.org on http://stsp.name
If you can identify concrete performance bottlenecks then I am more than happy to learn about them. There is a section in Got's README file which explains how the tool can be profiled. Some known bottlenecks exist and are fixable, it's just a matter of finding the time to work on them.
There was already an effort to improve CVS for use by OpenBSD, OpenCVS. This effort stalled, and if it had not stalled I probably would never have started working on Got, Maybe I would have contributed to OpenCVS instead.
"Making CVS more SVN-like" is a fool's errand, since that is what the SVN project itself is already doing. It's mission is to create a "compelling replacement for CVS". For various reasons SVN was never adopted by OpenBSD (even though, coincidentally, I have been an SVN developer since around the SVN 1.5 time frame).
Reply
Comments
By System Operator (46.10.166.16) on
Profiling, on FreeBSD and OpenBSD operating systems with both BSD and GNU userland tools and also GNU and LLVM
compilers and toolchain from base and also from ports, and the Linux kernel branches with GNU userland & utils
in both client and server mode for both CVS, SVN and Git is what you can certainly do and recommended oftener.
You can not alone in the Game of Trees project change the OpenBSD system kernel and userland tools as outlined
in the points of hardware acceleration, concurrency and parallel execution, file system and database workloads
and input/output performance. These are critical for all application servers, as you know well from the Apache
supra-project where SVN lives (along with the Java free world software tooling). These are C++ and APR tooling
mostly with Java being their second "application" language, after C++ being the implementation language. These
are the limiting premises of in OpenBSD system development, using these is strictly external (for now) applied
programming tooling. If you would like to address these details definitely you can add more Apache foundation.
We are all free from illusions about the distance between CVS to SVN to Git, these are quite similar toolkits.
Implementing a thin and OpenBSD specific efficient C and Shell tooling, for a repository independent client is
probably useful for both SVN and Git with the majority of publicly hosted software being vacuumed over to Git.
The server side however will not be that easy task and yuo will really hit grinding obstacles in OpenBSD base.
Profiling will not solve it, it requires the already outlined critical element of concurrency and performance.
The main benefit is going to be extending the OpenBSD SSH and CVS repository provisioning to add few desirable
capabilities and improving the OpenBSD HTTP server daemon in the base system, to become an application server,
integrating the SVN and easily Git protocols over HTTP & SSH transports. Certainly you can not achieve the Git
entire toolkit server side in the C and OpenBSD kernel facilities due to complexity and kernel specifics. It's
a Linux specific and C++ application server and tools, which just can't fit in the same parameters and epoch..
historically, backporting Git to an 80ies and 90ies computers is the really futile attempt that result in CVS.
Reply
Comments
By Stefan Sperling (stsp) stsp@openbsd.org on http://stsp.name
> You can not alone
> will not be that easy task
> Certainly you can not
> really futile attempt
OK, thank you for your concern and your kind consideration for my well-being and all the warnings about imminent doom and failure. I will be careful.
Reply
Comments
By System Operator (46.10.166.16) on
Well, it's obvious you have support and understanding on your work and it's interesting and useful. Challenges
can not stop you and words can not detract you. It's just very intriguing and meaningful to think about these.
If people had listened to enumeration of difficulties, much of the advent of software would not have happened.
Reply