IBM / FreeBSD Install problem

[Murray Taylor]

Server: IBM X3850 (88633SM)
CPU X 4: 40K2522
HDD X 6: 40K1051
IBM ServeRAID 8i: 39R8729

We are attempting to install FreeBSD 6.2-RELEASE onto this machine and
are running into a problem getting the operating system to recognise the
RAID controller. As a result not finding any disks when it comes to
installing the O/S.

We have attempted various modifications to the boot process, including
the loading of an "aac" module, which according to the BSD website,
should provide support for this type of controller.

When we attempt to boot to OS to install after making these above
modifications, the boot loader advises that this module already appears
to be loaded, which contradicts what I believe. In any respect, it
doesn't work either way (with or without the module manually loaded).

One side note (which i don't think is contributing) is that when I
attempt to start the boot loader with ACPI enabled, it freezes with the
message "cpu id 38 too high". However if I boot the boot loader with
ACPI disabled, this message dissapears. It _may_ be a possibility that a
bi-product of disabling the ACPI is causing the RAID controller to have
issues. This appears to be an issue because of the X4 CPU count ??

That's a quick summary of the problem we have, and the path(s) we have
been down to date to attempt to fix it. Any help you can provide would
be very much appreciated. We are at the position now where we are
prepared to pay for consulting services to get it going.

Dave Faulkner / Murray Taylor

Bytecraft Systems


--

"Any intelligent fool can make things bigger and more complex... It
takes a
touch of genius - and a lot of courage to move in the opposite
direction."
--Albert Einstein
---------------------------------------------------------------
The information transmitted in this e-mail is for the exclusive
use of the intended addressee and may contain confidential
and/or privileged material. Any review, re-transmission,
dissemination or other use of it, or the taking of any action
in reliance upon this information by persons and/or entities
other than the intended recipient is prohibited. If you
received this in error, please inform the sender and/or
addressee immediately and delete the material.

E-mails may not be secure, may contain computer viruses and
may be corrupted in transmission. Please carefully check this
e-mail (and any attachment) accordingly. No warranties are
given and no liability is accepted for any loss or damage
caused by such matters.
---------------------------------------------------------------

### This e-mail message has been scanned for Viruses by Bytecraft ###

[Jerry McAllister]

Post by Murray Taylor
Server: IBM X3850 (88633SM)
CPU X 4: 40K2522
HDD X 6: 40K1051
IBM ServeRAID 8i: 39R8729
We are attempting to install FreeBSD 6.2-RELEASE onto this machine and
are running into a problem getting the operating system to recognise the
RAID controller. As a result not finding any disks when it comes to
installing the O/S.
We have attempted various modifications to the boot process, including
the loading of an "aac" module, which according to the BSD website,
should provide support for this type of controller.

I have only installed on a couple of raids and so don't know about
them all or even this one. So, this might not apply to your situation.
But, I found that I had to study DMESG very carefully to find out what
device to use for them. The system seemed to put out a lot of messages
that looked like other device names but in the end there was just one
little line that pointed to the correct one. The most recent one was
a Dell Perc 3i or something like that and I had to run the fixit and
study the boot messages to figure it out. I don't have that one
available to look at what it turned out to be, but it was more simple
than I first thought from all the stuff it wrote out. After mucking
with fixit a bit, then sysinstall seemed to figure it out OK. I don't
remember actually changing anything - just fishing around a while.
I may have run fdisk under fixit to look at things and maybe delete
some slices.

So, rather than trying to change things right off, I would suggest
looking carefully at stuff and trying to determine what it is already
doing.

Anyway, good luck,

////jerry

Post by Murray Taylor
When we attempt to boot to OS to install after making these above
modifications, the boot loader advises that this module already appears
to be loaded, which contradicts what I believe. In any respect, it
doesn't work either way (with or without the module manually loaded).
One side note (which i don't think is contributing) is that when I
attempt to start the boot loader with ACPI enabled, it freezes with the
message "cpu id 38 too high". However if I boot the boot loader with
ACPI disabled, this message dissapears. It _may_ be a possibility that a
bi-product of disabling the ACPI is causing the RAID controller to have
issues. This appears to be an issue because of the X4 CPU count ??
That's a quick summary of the problem we have, and the path(s) we have
been down to date to attempt to fix it. Any help you can provide would
be very much appreciated. We are at the position now where we are
prepared to pay for consulting services to get it going.
Dave Faulkner / Murray Taylor
Bytecraft Systems
--
"Any intelligent fool can make things bigger and more complex... It
takes a
touch of genius - and a lot of courage to move in the opposite
direction."
--Albert Einstein
---------------------------------------------------------------
The information transmitted in this e-mail is for the exclusive
use of the intended addressee and may contain confidential
and/or privileged material. Any review, re-transmission,
dissemination or other use of it, or the taking of any action
in reliance upon this information by persons and/or entities
other than the intended recipient is prohibited. If you
received this in error, please inform the sender and/or
addressee immediately and delete the material.
E-mails may not be secure, may contain computer viruses and
may be corrupted in transmission. Please carefully check this
e-mail (and any attachment) accordingly. No warranties are
given and no liability is accepted for any loss or damage
caused by such matters.
---------------------------------------------------------------
### This e-mail message has been scanned for Viruses by Bytecraft ###
_______________________________________________
http://lists.freebsd.org/mailman/listinfo/freebsd-questions

[Dag-Erling Smørgrav]

Post by Murray Taylor
We are attempting to install FreeBSD 6.2-RELEASE onto this machine and
are running into a problem getting the operating system to recognise the
RAID controller. As a result not finding any disks when it comes to
installing the O/S.
We have attempted various modifications to the boot process, including
the loading of an "aac" module, which according to the BSD website,
should provide support for this type of controller.

Excuse me for asking a stupid question, but did you define an array
before attempting to install FreeBSD? The aac driver won't attach
individual disks, it will only attached defined arrays.

Post by Murray Taylor
When we attempt to boot to OS to install after making these above
modifications, the boot loader advises that this module already appears
to be loaded, which contradicts what I believe. In any respect, it
doesn't work either way (with or without the module manually loaded).

The loader is correct, aac is included in GENERIC.

Post by Murray Taylor
One side note (which i don't think is contributing) is that when I
attempt to start the boot loader with ACPI enabled, it freezes with the
message "cpu id 38 too high". However if I boot the boot loader with
ACPI disabled, this message dissapears. It _may_ be a possibility that a
bi-product of disabling the ACPI is causing the RAID controller to have
issues. This appears to be an issue because of the X4 CPU count ??

It appears the server uses non-consecutive CPU numbers, and we use a
static array with 32 slots, indexed by CPU number, to hold information
about the CPUs (or rather the local APICs they contain).

If you can either install without ACPI, or remove two of the CPUs
during installation, this should be fairly easy to fix: change the
definition of NLAPICS in /usr/src/sys/{amd64,i386}/acpica/madt.c and
rebuild your kernel, then boot with ACPI enabled and report back to
us.

DES

[Mark Tinguely]

Post by Dag-Erling Smørgrav
If you can either install without ACPI, or remove two of the CPUs
during installation, this should be fairly easy to fix: change the
definition of NLAPICS in /usr/src/sys/{amd64,i386}/acpica/madt.c and
rebuild your kernel, then boot with ACPI enabled and report back to
us.
DES
--=20

I suggested that in email too, but looking closer, I think the MAXCPU
needs to be increased because the cpu number uses the apic_id. Or could
that be changed with a logical CPU to APIC ID lookup?

Isn't the APIC IDs programmable? not that I am suggesting that, I
can think of headaches of all the places (like interrupt tables)
where it needs to be changed, not to mention the worry that the
lower APIC IDs were assigned to IOAPICs.

--Mark Tinguely

[Dag-Erling Smørgrav]

Post by Mark Tinguely
I suggested that in email too, but looking closer, I think the MAXCPU
needs to be increased because the cpu number uses the apic_id. Or could
that be changed with a logical CPU to APIC ID lookup?
Isn't the APIC IDs programmable? not that I am suggesting that, I
can think of headaches of all the places (like interrupt tables)
where it needs to be changed, not to mention the worry that the
lower APIC IDs were assigned to IOAPICs.

I don't know, you'd have to ask jhb@ about the details.

DES

[John Baldwin]

Post by Mark Tinguely
I suggested that in email too, but looking closer, I think the MAXCPU
needs to be increased because the cpu number uses the apic_id. Or could
that be changed with a logical CPU to APIC ID lookup?
Isn't the APIC IDs programmable? not that I am suggesting that, I
can think of headaches of all the places (like interrupt tables)
where it needs to be changed, not to mention the worry that the
lower APIC IDs were assigned to IOAPICs.

APIC IDs are not programmable (well, they are on I/O APICs, but not local
APICs). However, I am working on patches to support all valid APIC IDs for
both mptable and MADT. Bumping up NLAPICS as a temporary workaround should
suffice for now.

[Mark Tinguely]

APIC IDs are not programmable (well, they are on I/O APICs, but not local=20
APICs). However, I am working on patches to support all valid APIC IDs for=
=20
both mptable and MADT. Bumping up NLAPICS as a temporary workaround should=
=20
suffice for now.
=2D-=20
John Baldwin

IMO, the quick solution also requires that MAX_APICID in
[amd64/amd64 | i386/i386]/local_apic.c needs to be changed
because lapic_create() checks if the passed apic_id > MAX_APICID.

Also in [amd64/amd64 | i386/i386]/mp_machdep.c checks in cpu_add()
if the passed apic_id >= MAXCPU. There are a couple other checks
in mp_machdep.c before converting to use the cpu_apic_ids[] array.

I was curious, and wrote up a patch file with the potential minor changes
for -current at http://www.casselton.com/~tinguely/acpicid.patch .
I saw one more change needed to use on FreeBSD 6.2-RELEASE.

--Mark Tinguely

[Murray Taylor]

Thanks all,

We will look into the code editting and see what we can get
however we are on an very short time frame so may not be able to
slot it in before a maintenance slot where we need to be able to drop in
the box 'seamlessly' .....

But I have noted the proposed fix provided on these lists into the red
book!

mjt

-----Original Message-----
Sent: Friday, 20 April 2007 3:37 AM
Subject: Re: IBM / FreeBSD Install problem

Post by Dag-Erling Smørgrav
If you can either install without ACPI, or remove two of the CPUs
during installation, this should be fairly easy to fix: change the
definition of NLAPICS in

/usr/src/sys/{amd64,i386}/acpica/madt.c and

Post by Dag-Erling Smørgrav
rebuild your kernel, then boot with ACPI enabled and report back to
us.
DES
--=20

I suggested that in email too, but looking closer, I think the MAXCPU
needs to be increased because the cpu number uses the
apic_id. Or could
that be changed with a logical CPU to APIC ID lookup?
Isn't the APIC IDs programmable? not that I am suggesting that, I
can think of headaches of all the places (like interrupt tables)
where it needs to be changed, not to mention the worry that the
lower APIC IDs were assigned to IOAPICs.
--Mark Tinguely

---------------------------------------------------------------
The information transmitted in this e-mail is for the exclusive
use of the intended addressee and may contain confidential
and/or privileged material. Any review, re-transmission,
dissemination or other use of it, or the taking of any action
in reliance upon this information by persons and/or entities
other than the intended recipient is prohibited. If you
received this in error, please inform the sender and/or
addressee immediately and delete the material.

E-mails may not be secure, may contain computer viruses and
may be corrupted in transmission. Please carefully check this
e-mail (and any attachment) accordingly. No warranties are
given and no liability is accepted for any loss or damage
caused by such matters.
---------------------------------------------------------------

### This e-mail message has been scanned for Viruses by Bytecraft ###

[Murray Taylor]

Thanks Jerry,

We have determined that our problem is related to the
god-awful mess known as ACPI.

With a single processor installed, it all boots fine, including
finding PCI busses 1 and 2 .... and the raid on the aac driver is
peachy!

With 2 or more processors installed, and ACPI enabled, it panics
with a madt error about id 38 is greater than the allowed max

And with ACPI disabled, it boots, but doesnt find PCI busses 1 and 2,
which is unfortunate as the RAID controller sits in PCI bus 1 ....
and the bge inet interface seems to be on PCI bus 2 ....

So at present we are on the FC6 path, which is obviously much
softer on the vagaries of ACPI and is ignoring the crap data returns.

mjt

-----Original Message-----
Sent: Friday, 20 April 2007 12:40 AM
To: Murray Taylor
Subject: Re: IBM / FreeBSD Install problem

Post by Murray Taylor
Server: IBM X3850 (88633SM)
CPU X 4: 40K2522
HDD X 6: 40K1051
IBM ServeRAID 8i: 39R8729
We are attempting to install FreeBSD 6.2-RELEASE onto this

machine and

Post by Murray Taylor
are running into a problem getting the operating system to

recognise the

Post by Murray Taylor
RAID controller. As a result not finding any disks when it comes to
installing the O/S.
We have attempted various modifications to the boot

process, including

Post by Murray Taylor
the loading of an "aac" module, which according to the BSD website,
should provide support for this type of controller.

I have only installed on a couple of raids and so don't know about
them all or even this one. So, this might not apply to your
situation.
But, I found that I had to study DMESG very carefully to find
out what
device to use for them. The system seemed to put out a lot
of messages
that looked like other device names but in the end there was just one
little line that pointed to the correct one. The most
recent one was
a Dell Perc 3i or something like that and I had to run the fixit and
study the boot messages to figure it out. I don't have that one
available to look at what it turned out to be, but it was more simple
than I first thought from all the stuff it wrote out. After mucking
with fixit a bit, then sysinstall seemed to figure it out OK. I don't
remember actually changing anything - just fishing around a while.
I may have run fdisk under fixit to look at things and maybe delete
some slices.
So, rather than trying to change things right off, I would suggest
looking carefully at stuff and trying to determine what it is already
doing.
Anyway, good luck,
////jerry

Post by Murray Taylor
When we attempt to boot to OS to install after making these above
modifications, the boot loader advises that this module

already appears

Post by Murray Taylor
to be loaded, which contradicts what I believe. In any respect, it
doesn't work either way (with or without the module

manually loaded).

Post by Murray Taylor
One side note (which i don't think is contributing) is that when I
attempt to start the boot loader with ACPI enabled, it

freezes with the

Post by Murray Taylor
message "cpu id 38 too high". However if I boot the boot loader with
ACPI disabled, this message dissapears. It _may_ be a

possibility that a

Post by Murray Taylor
bi-product of disabling the ACPI is causing the RAID

controller to have

Post by Murray Taylor
issues. This appears to be an issue because of the X4 CPU count ??
That's a quick summary of the problem we have, and the

path(s) we have

Post by Murray Taylor
been down to date to attempt to fix it. Any help you can

provide would

Post by Murray Taylor
be very much appreciated. We are at the position now where we are
prepared to pay for consulting services to get it going.
Dave Faulkner / Murray Taylor
Bytecraft Systems
--
"Any intelligent fool can make things bigger and more complex... It
takes a
touch of genius - and a lot of courage to move in the opposite
direction."
--Albert Einstein
---------------------------------------------------------------
The information transmitted in this e-mail is for the exclusive
use of the intended addressee and may contain confidential
and/or privileged material. Any review, re-transmission,
dissemination or other use of it, or the taking of any action
in reliance upon this information by persons and/or entities
other than the intended recipient is prohibited. If you
received this in error, please inform the sender and/or
addressee immediately and delete the material.
E-mails may not be secure, may contain computer viruses and
may be corrupted in transmission. Please carefully check this
e-mail (and any attachment) accordingly. No warranties are
given and no liability is accepted for any loss or damage
caused by such matters.
---------------------------------------------------------------
### This e-mail message has been scanned for Viruses by

Bytecraft ###

Post by Murray Taylor
_______________________________________________
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to

---------------------------------------------------------------
The information transmitted in this e-mail is for the exclusive
use of the intended addressee and may contain confidential
and/or privileged material. Any review, re-transmission,
dissemination or other use of it, or the taking of any action
in reliance upon this information by persons and/or entities
other than the intended recipient is prohibited. If you
received this in error, please inform the sender and/or
addressee immediately and delete the material.

E-mails may not be secure, may contain computer viruses and
may be corrupted in transmission. Please carefully check this
e-mail (and any attachment) accordingly. No warranties are
given and no liability is accepted for any loss or damage
caused by such matters.
---------------------------------------------------------------

### This e-mail message has been scanned for Viruses by Bytecraft ###

[John Baldwin]

Post by John Baldwin
APIC IDs are not programmable (well, they are on I/O APICs, but not

local=20

Post by John Baldwin
APICs). However, I am working on patches to support all valid APIC IDs

for=

Post by John Baldwin
=20
both mptable and MADT. Bumping up NLAPICS as a temporary workaround

should=

Post by John Baldwin
=20
suffice for now.
=2D-=20
John Baldwin

IMO, the quick solution also requires that MAX_APICID in
[amd64/amd64 | i386/i386]/local_apic.c needs to be changed
because lapic_create() checks if the passed apic_id > MAX_APICID.
Also in [amd64/amd64 | i386/i386]/mp_machdep.c checks in cpu_add()
if the passed apic_id >= MAXCPU. There are a couple other checks
in mp_machdep.c before converting to use the cpu_apic_ids[] array.
I was curious, and wrote up a patch file with the potential minor changes
for -current at http://www.casselton.com/~tinguely/acpicid.patch .
I saw one more change needed to use on FreeBSD 6.2-RELEASE.

What I have so far is somewhat similar, but goes ahead and allows the full
range of APIC IDs while trying to still honor MAXCPU correctly. I haven't
ported it to i386 yet, nor compiled it yet, much less booted it. :) I hope
to at least get it booted on amd64 today.