Contributed by paul on from the Enterprise OpenBSD dept.
Mark writes:
This one was a bit of a bugger. I started hacking on it during the c2k8 hackathon. Paul Greidanus had set me up with access to the system controller, a nice fast v210 to compile my kernels, and a private network between them such that I could netboot my kernels. However the machine didn't even like our bootloader. This was quite worrying since in the past the bootloader was the part that always just worked. Even for supporting the new sun4v architecture I didn't have to make changes to it.
I noticed that after resetting the machine (which takes quite a while on such a big box) it greeted me with:
Sun Fire V1280 OpenFirmware version 5.17.0 (03/05/04 11:28) Copyright 2001-2004 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. SmartFirmware, Copyright (C) 1996-2001. All rights reserved. 98304 MB memory installed, Serial #54879917. Ethernet address 0:3:ba:45:66:ad, Host ID: 834566ad.
WTF? SmartFirmware? OpenFirmware 5.x? Even Sun's latest offerings use 4.x. This machine has seriously weird firmware!
After fixing some issues in the bootloader I had it load a kernel, and it actually printed a first message on the console. After that, I hit a brick wall. As soon as I tried taking over control from the firmware completely, the machine would just reset. With a few cheats I got it to print a copyright message and after some more hacking I even got it to probe all devices. But it wouldn't run any userland code. Instead it crashed ending up in a seemingly impossible state. When I left Edmonton it was still in this rather sorry state.
I spent most of the three weeks after the hackathon with travel, hiking and recovering from jetlag. But last week I decided to have another go at the v1280. Looking at the problem again with a fresh mind helped, because I soon realized that the stupid firmware was playing tricks behind my back. After changing the register window handling to be "compatible" with Solaris the machine was much happier, and booted multi-user soon. Meanwhile I had accumulated quite a number of changes to placate the firmware. Testing these changes on other machines took a bit of time, but I finally got them all in the tree last weekend.
console is /sgcn Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. Copyright (c) 1995-2008 OpenBSD. All rights reserved. http://www.OpenBSD.org OpenBSD 4.4-beta (GENERIC.MP) #27: Sun Jul 6 01:44:32 MDT 2008 root@v1280_head.cein.ualberta.ca:/usr/src/sys/arch/sparc64/compile/GENERIC.MP real mem = 103079215104 (98304MB) avail mem = 101204992000 (96516MB) mainbus0 at root: Sun Fire V1280 ssm0 at mainbus0 cpu0 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu0: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu1 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu1: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu2 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu2: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu3 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu3: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu4 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu4: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu5 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu5: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu6 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu6: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu7 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu7: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu8 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu8: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu9 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu9: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu10 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu10: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured cpu11 at ssm0: SUNW,UltraSPARC-III+ (rev 2.3) @ 900 MHz cpu11: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external (512 b/l) "memory-controller" at ssm0 not configured schizo0 at ssm0: "Schizo", version 4, ign 600, bus B 0 to 0 schizo0: dvma map c0000000-ffffffff, iotdb 17080000-17180000 pci0 at schizo0 sbbc0 at pci0 dev 4 function 0 "Sun SBBC" rev 0x02 pciide0 at pci0 dev 3 function 0 "CMD Technology PCI0646" rev 0x07: DMA, channel 0 configured to native-PCI, channel 1 configured to native-PCI pciide0: using ivec 0x608 for native-PCI interrupt atapiscsi0 at pciide0 channel 0 drive 0 scsibus0 at atapiscsi0: 2 targets, initiator 7 cd0 at scsibus0 targ 0 lun 0:ATAPI 5/cdrom removable cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 pciide0: channel 1 ignored (disabled) schizo1 at ssm0: "Schizo", version 4, ign 600, bus A 0 to 0 schizo1: dvma map c0000000-ffffffff, iotdb 17184000-17284000 pci1 at schizo1 siop0 at pci1 dev 2 function 0 "Symbios Logic 53c1010-66" rev 0x01: ivec 0x61c, using 8K of on-board RAM scsibus1 at siop0: 16 targets, initiator 7 sd0 at scsibus1 targ 0 lun 0: SCSI3 0/direct fixed sd0: 34732MB, 24622 cyl, 27 head, 107 sec, 512 bytes/sec, 71132959 sec total sd1 at scsibus1 targ 1 lun 0: SCSI3 0/direct fixed sd1: 34732MB, 24622 cyl, 27 head, 107 sec, 512 bytes/sec, 71132959 sec total siop1 at pci1 dev 2 function 1 "Symbios Logic 53c1010-66" rev 0x01: ivec 0x61d, using 8K of on-board RAM scsibus2 at siop
There is still a bit of work to do. These machines don't have a traditional console. Instead they communicate with a system controller through a bit of shared memory. Writing on the console is done by writing the output into shared memory and pinging the system controller. When there is console input, the system controller pings us and we read it from a different bit of shared memory. This doesn't work yet, but for now the firmware console works fine.
Now that the Sun Fire V1280 is supported, OpenBSD will almost certainly run on the Sun Fire 3800/4800/4810/6800 series. There's a fair chance it will also run on the Sun Fire 12K/15K and Sun Fire E2900/E4900/E6900 series. We may even run on the Sun Fire E20K/E25K now! Most of these machines can be partitioned into multiple domains, I can imagine that people may be interested in running an OpenBSD firewall on a small partition in the same box. If you have one of these boxes and a partition that you can play with, it would be nice if you could give a new snapshot a spin on it and tell me about the results. But even if you're running Solaris, you could help me out by sending me the output of "prtconf -pv" for the box.Great work Mark, keep it up!
(Comments are closed)
By David Chisnall (82.7.199.50) on
Comments
By mk (193.128.72.68) on
It depends on what you're doing on it. If you don't do a lot of IO etc., it'll probably work fairly well for you.
Comments
By Anonymous Coward (96.21.15.58) on
>
> It depends on what you're doing on it. If you don't do a lot of IO etc., it'll probably work fairly well for you.
So, just surfing the web, with lynx is ok? ;-)
By Anonymous Coward (72.14.20.144) on
Unfortunately, there is almost no serious benefit unless it's strictly userland and not heavily I/O dependent, at least not until rthreads is stable. =(
Comments
By Anonymous Coward (198.175.14.193) on
>
> Unfortunately, there is almost no serious benefit unless it's strictly userland and not heavily I/O dependent, at least not until rthreads is stable. =(
>
That's only true if MySQL is your big app... For instance, PostgreSQL already scales across multiple CPUs under the current model, so does gcc with make -j, apache with multiple processes, etc.
As the kernel moves to get partitioned out with fine-grained locking and starts to run on multiple CPUs, that will be the biggest benefit for some users. For others who aren't yet maxing out the first CPU with the kernel, they can take advantage today. For MySQL, or other heavily threaded apps, rthreads is important.
FreeBSD's version 7 benchmarkers found that MySQL's thread model is poorly constructed and can't deal with locking contention beyond 16 CPUs. They found that PostgreSQL, which uses no threads at all, just traditional message passing between processes, scales far beyond 16 CPUs. Libthr, their current threading library, is very similar in concept to rthreads (mapping one process per thread.) Rthreads isn't the solution to everything...but it will be nice for some users.
Comments
By Anonymous Coward (72.14.20.144) on
> That's only true if MySQL is your big app... For instance, PostgreSQL already scales across multiple CPUs under the current model, so does gcc with make -j, apache with multiple processes, etc.
<snip the blabber about MySQL vs. PostgreSQL>
So why even initiate the changes proposed by rthreads? The bottom line is that you can make a few valid points in explicit cases, but that's the extent of the benefit. I can name just as many shortcomings of the current model, not the least of which have forced me to use other platforms for specific applications.
IMHO, the bottom line is that more users need to enable rthreads on their own platforms to push feedback to the development team so that the giant [Lock] monkey can be cast off our collective backs.
Comments
By Brad (206.51.28.2) brad at comstyle dot com on
rthreads is not at the point where it needs feedback yet. It is still very experimental and needs a lot of work which hopefully should change quite a bit after the 4.4 release. At the moment there are way too many bugs and unresolved issues to realistically use rthreads.
By Damien Miller (djm) on http://www.mindrot.org/~djm/
> rthreads on their own platforms to push feedback to the
> development team so that the giant [Lock] monkey can be
> cast off our collective backs.
rthreads and biglock are almost completely orthogonal. Finishing rthreads won't magically make the kernel reentrant, and making the kernel reentrant won't fix userland threads.
By Anonymous Coward (66.39.160.90) on
> So why even initiate the changes proposed by rthreads? The bottom line is that you can make a few valid points in explicit cases, but that's the extent of the benefit. I can name just as many shortcomings of the current model, not the least of which have forced me to use other platforms for specific applications.
>
> IMHO, the bottom line is that more users need to enable rthreads on their own platforms to push feedback to the development team so that the giant [Lock] monkey can be cast off our collective backs.
Rthreads has absolutely nothing what-so-ever to do with the kernel giant lock.
How can you see shortcomings in the current model when you have no idea what you're talking about?
Rthreads isn't in a working state yet. Once it works and fully implements a POSIX thread environment with working signals and other common unix features, then we can talk about shortcomings. Right now the shortcoming is that you can't even fucking use it. And, people are working on that.
Comments
By Gimlet (66.167.69.18) on
Oh great, it's a GNU project.
By Anonymous Coward (24.91.217.63) on
Comments
By Anonymous Coward (66.39.160.90) on
>
Too many morons who don't have the background to actually understand technical issues or make coherent arguments are the ones who spend their worthless time moderating. I suspect a majority who actually have some clue will, at a minimum, not spend their time moderating. So you end up with jingoistic ideology working at a base level against Al Qaeda's anti-OpenBSD messages. Fight on, soldiers.
By Anonymous Coward (89.77.162.243) on
>
So -1/3 is oblivion eh?
OMG... it's full of... moderated comments! Repent now!
ps. will you chill? most people here have lives and better shit to do than modding...
By Motley Fool (MotleyFool) motleyfool@dieselrepower.org on
By 1337 (75.216.199.145) on
You may think I'm kidding, but I'm really not. It's good to know, however that OpenBSD is a little ahead of the curve and maybe the current big iron hardware can help tune scheduling algorithms accordingly, so when these hit our machine rooms or desks, we'll have some serious computing power... to run bittorent and nethack :)
No, seriously, this is a great accomplishment, while I highly doubt lots of companies will be running OpenBSD on this $300k monster, it's still very neat. I'd love to see some benchmarks on this box, especially vs. Solaris (and sun cc).
A very nice box with a very nice operating system, yet a little noisy for my apartment.
Mia
Comments
By Anonymous Coward (2a01:348:108:155:216:41ff:fe53:6a45) on
Read the article about "Most of these machines can be partitioned..." for one possible use.