fixing IRQ storms

[Mike Tancsa]

Using
http://lists.freebsd.org/pipermail/freebsd-bugs/2007-March/022753.html

I have been able to get my GPRS card to be recognized in
FreeBSD. However, the IRQ it has picked, seems to conflict with the
irq of the cardbus causing an interrupt storm. Is there a way to
make the card use a different IRQ ? I tried ORing in
PCCARD_CFE_IRQSHARE, but it doesnt seem to make a difference.


The quirks entry shows from the above PR is and I tried adding in
PCCARD_CFE_IRQSHARE without luck. It always goes to irq 10 and it
always causes a storm. Any ideas how to fix it ? Thanks for any pointers!


static struct pccard_config_entry pccard_sierra_a555_func1_cfe0 = {
0x22, /* cfe number */
PCCARD_CFE_IO8 | PCCARD_CFE_IRQLEVEL,
PCCARD_IFTYPE_IO,
1, /* num_iospace */
0, /* iomask */
{ { 0x0008, 0x3e8 } }, /* iospace */
0x3fbc, /* irqmask */
0, /* num_memspace */
{ }, /* memspace */
0, /* maxtwins */
};

Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Enhanced Am486DX4/Am5x86 Write-Back (486-class CPU)
Origin = "AuthenticAMD" Id = 0x494 Stepping = 4
Features=0x1<FPU>
real memory = 67108864 (64 MB)
avail memory = 60379136 (57 MB)
cpu0 on motherboard
sysctl machdep.i8254_freq=1189161 returns 0
Timecounter "ELAN" frequency 8333333 Hz quality 1000
pcib0: <AMD Elan SC520 host to PCI bridge> pcibus 0 on motherboard
pci0: <PCI bus> on pcib0
cbb0: <TI1410 PCI-CardBus Bridge> mem 0xa0000000-0xa0000fff irq 10 at
device 9.0 on pci0
cardbus0: <CardBus bus> on cbb0
pccard0: <16-bit PCCard bus> on cbb0
sis0: <NatSemi DP8381[56] 10/100BaseTX> port 0xe100-0xe1ff mem
0xa0001000-0xa0001fff irq 11 at device 18.0 on pci0
sis0: Silicon Revision: DP83816A
miibus0: <MII bus> on sis0
ukphy0: <Generic IEEE 802.3u media interface> on miibus0
ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
sis0: Ethernet address: 00:00:24:c5:75:10
sis1: <NatSemi DP8381[56] 10/100BaseTX> port 0xe200-0xe2ff mem
0xa0002000-0xa0002fff irq 5 at device 19.0 on pci0
sis1: Silicon Revision: DP83816A
miibus1: <MII bus> on sis1
ukphy1: <Generic IEEE 802.3u media interface> on miibus1
ukphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
sis1: Ethernet address: 00:00:24:c5:75:11
isa0: <ISA bus> on motherboard
orm0: <ISA Option ROM> at iomem 0xc8000-0xd0fff on isa0
ata0 at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0
ata1 at port 0x170-0x177,0x376 irq 15 on isa0
uart0: <16550 or compatible> at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
uart0: console (19200,n,8,1)
Timecounters tick every 1.000 msec
Elan-mmcr driver: MMCR at 0xc5b71000.
Elan-mmcr Soekris net45xx comBIOS ver. 1.28 20050527 Copyright (C) 2000-2005
uart2: <Sierra Wireless AC860> at port 0x3e8-0x3ef irq 10 function 1
config 34 on pccard0
Interrupt storm detected on "irq10:"; throttling interrupt source
ad0: 977MB <Hitachi XX.V.3.4.0.0 Rev 0.00> at ata0-master PIO4

% vmstat -i
interrupt total rate
irq0: clk 151537 996
irq4: uart0 288 1
irq8: rtc 19395 127
irq10: cbb0 uart2 134314 883
irq11: sis0 192 1
irq14: ata0 16891 111
Total 322617 2122

---Mike


--------------------------------------------------------------------
Mike Tancsa, tel [phone]
Sentex Communications, ***@sentex.net
Providing Internet since 1994 www.sentex.net
Cambridge, Ontario Canada www.sentex.net/mike

[Mike Tancsa]

Post by Mike Tancsa
Using
http://lists.freebsd.org/pipermail/freebsd-bugs/2007-March/022753.html

Tried with sio, but the card is not usable due to all the overflows.
(e.g. just ati3 and only part of the info gets displayed)


isa0: <ISA bus> on motherboard
orm0: <ISA Option ROM> at iomem 0xc8000-0xd0fff on isa0
ata0 at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0
ata1 at port 0x170-0x177,0x376 irq 15 on isa0
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A, console
Timecounters tick every 1.000 msec
Elan-mmcr driver: MMCR at 0xc5b71000.
Elan-mmcr Soekris net45xx comBIOS ver. 1.28 20050527 Copyright (C) 2000-2005
sio2: <Sierra Wireless AC860> at port 0x3e8-0x3ef irq 10 function 1
config 34 on pccard0
sio2: type 16550A
sio2: unable to activate interrupt in fast mode - using normal mode
ad0: 977MB <Hitachi XX.V.3.4.0.0 Rev 0.00> at ata0-master PIO4
Trying to mount root from ufs:/dev/ad0s1a
sio2: 1 more interrupt-level buffer overflow (total 1)
sio2: 108 more interrupt-level buffer overflows (total 109)
sio2: 108 more interrupt-level buffer overflows (total 217)
sio2: 108 more interrupt-level buffer overflows (total 325)
sio2: 109 more interrupt-level buffer overflows (total 434)


---Mike

Post by Mike Tancsa
--------------------------------------------------------------------
Mike Tancsa, tel [phone]
Providing Internet since 1994 www.sentex.net
Cambridge, Ontario Canada www.sentex.net/mike

[Mike Tancsa]

Mike,
have a look at GNAT: http://www.freebsd.org/cgi/query-pr.cgi?pr=51982
You need to patch your kernel sources a bit (all info in the PR) and
your silo overflows will be gone. I've done that to get a Merlin
U630 working (w/o patching sio.c the card was almost unusable).

Hi,
Thanks for the note. Actually, I have done this in the past
(I am even in the above PR :)), but I had found using the UART driver
instead of sio was better for working around such issues on the
whole. The overflow rate is so high in the case, even increasing the
ticks by 10 doesnt seem to fix the issue. It seems to reduce the
overflows a bit, but nowhere nearly enough

sio2: 108 more interrupt-level buffer overflows (total 108)
sio2: 81 more interrupt-level buffer overflows (total 189)
sio2: 108 more interrupt-level buffer overflows (total 297)
sio2: 81 more interrupt-level buffer overflows (total 378)
sio2: 33 more interrupt-level buffer overflows (total 411)
sio2: 76 more interrupt-level buffer overflows (total 487)

With the uart driver, I am able to talk to the modem OK, but the cpu
is really high dealing with all those interrupts.

I was hoping I could just tell the card to use a different interrupt
like 5 which is available, but I dont know how to do that, or if its
even possible.

---Mike

[Volker]

Post by Mike Tancsa

Post by Mike Tancsa
Using
http://lists.freebsd.org/pipermail/freebsd-bugs/2007-March/022753.html

Tried with sio, but the card is not usable due to all the overflows.
(e.g. just ati3 and only part of the info gets displayed)
isa0: <ISA bus> on motherboard
orm0: <ISA Option ROM> at iomem 0xc8000-0xd0fff on isa0
ata0 at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0
ata1 at port 0x170-0x177,0x376 irq 15 on isa0
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A, console
Timecounters tick every 1.000 msec
Elan-mmcr driver: MMCR at 0xc5b71000.
Elan-mmcr Soekris net45xx comBIOS ver. 1.28 20050527 Copyright (C)
2000-2005
sio2: <Sierra Wireless AC860> at port 0x3e8-0x3ef irq 10 function 1
config 34 on pccard0
sio2: type 16550A
sio2: unable to activate interrupt in fast mode - using normal mode
ad0: 977MB <Hitachi XX.V.3.4.0.0 Rev 0.00> at ata0-master PIO4
Trying to mount root from ufs:/dev/ad0s1a
sio2: 1 more interrupt-level buffer overflow (total 1)
sio2: 108 more interrupt-level buffer overflows (total 109)
sio2: 108 more interrupt-level buffer overflows (total 217)
sio2: 108 more interrupt-level buffer overflows (total 325)
sio2: 109 more interrupt-level buffer overflows (total 434)

Mike,

have a look at GNAT: http://www.freebsd.org/cgi/query-pr.cgi?pr=51982

You need to patch your kernel sources a bit (all info in the PR) and
your silo overflows will be gone. I've done that to get a Merlin
U630 working (w/o patching sio.c the card was almost unusable).

HTH,

Volker

[Volker]

Mike,
have a look at GNAT: http://www.freebsd.org/cgi/query-pr.cgi?pr=51982
You need to patch your kernel sources a bit (all info in the PR) and
your silo overflows will be gone. I've done that to get a Merlin
U630 working (w/o patching sio.c the card was almost unusable).

Hi,
Thanks for the note. Actually, I have done this in the past (I
am even in the above PR :)), but I had found using the UART driver
instead of sio was better for working around such issues on the whole.
The overflow rate is so high in the case, even increasing the ticks by
10 doesnt seem to fix the issue. It seems to reduce the overflows a bit,
but nowhere nearly enough
sio2: 108 more interrupt-level buffer overflows (total 108)
sio2: 81 more interrupt-level buffer overflows (total 189)
sio2: 108 more interrupt-level buffer overflows (total 297)
sio2: 81 more interrupt-level buffer overflows (total 378)
sio2: 33 more interrupt-level buffer overflows (total 411)
sio2: 76 more interrupt-level buffer overflows (total 487)
With the uart driver, I am able to talk to the modem OK, but the cpu is
really high dealing with all those interrupts.
I was hoping I could just tell the card to use a different interrupt
like 5 which is available, but I dont know how to do that, or if its
even possible.
---Mike

Mike,

I've played with uart(4) a while (~6mo?) back (for use with a Merlin
3G/GPRS card) and had even more trouble. Unfortunately I can't
remember what errors I was experiencing with but I've had a short
conversation with the uart maintainer.

If my memories are right, the maintainer stated it will take some
time to fix the uart problems. Haven't tried that since. But IMO the
design of uart is definitely cleaner compared to sio.

HTH,

Volker

[Dinesh Nair]

You need to patch your kernel sources a bit (all info in the PR) and
your silo overflows will be gone. I've done that to get a Merlin

i was about to comment on that, given that i extensively use a Huawei
3G/UMTS card without experiencing any of the symptoms before i realized
that i've got HZ=1000 in my kernel config.

[Mike Tancsa]

Post by Mike Tancsa
Using
http://lists.freebsd.org/pipermail/freebsd-bugs/2007-March/022753.html
I have been able to get my GPRS card to be recognized in
FreeBSD. However, the IRQ it has picked, seems to conflict with the
irq of the cardbus causing an interrupt storm. Is there a way to
make the card use a different IRQ ? I tried ORing in
PCCARD_CFE_IRQSHARE, but it doesnt seem to make a difference.

Just to followup on this thread in case others run into this
problem. The card is "working" with the uart driver, but I am still
getting an initial interrupt storm at boot up....


...from a boot -v snippet


Elan-mmcr driver: MMCR at 0xc5b53000.
Elan-mmcr Soekris net45xx comBIOS ver. 1.28 20050527 Copyright (C) 2000-2005
pflog0: bpf attached
lo0: bpf attached
pccard0: using CIS quirks for Sierra Wireless, AC860, 3G Network Adapter, R1
pccard0: CIS version PC Card Standard 7.0
pccard0: CIS info: Sierra Wireless, AC860, 3G Network Adapter, R1
pccard0: Manufacturer code 0x192, product 0x710
pccard0: function 1: serial port, ccr addr 700 mask 73
pccard0: function 1, config table entry 34: I/O card; irq mask 3fbc;
iomask 0, iospace 3e8-3ef; io8 irqlevel
uart4: <Sierra Wireless AC860> at port 0x3e8-0x3ef irq 10 function 1
config 34 on pccard0
Interrupt storm detected on "irq10:"; throttling interrupt source
ata0-master: pio=PIO4 wdma=WDMA2 udma=UNSUPPORTED cable=40 wire
ad0: 991MB <ELITE PRO CF CARD 1GB 20070205> at ata0-master PIO4
ad0: 2030112 sectors [2014C/16H/63S] 1 sectors/interrupt 1 depth queue


And looking at vmstat -i

[datawest-soekris]# vmstat -i ;sleep 1;vmstat -i
interrupt total rate
irq0: clk 137079 993
irq4: uart0 502 3
irq8: rtc 17544 127
irq10: cbb0 uart4 128357 930
irq11: sis0 29 0
irq14: ata0 16188 117
Total 299699 2171
interrupt total rate
irq0: clk 138212 994
irq4: uart0 529 3
irq8: rtc 17689 127
irq10: cbb0 uart4 129477 931
irq11: sis0 29 0
irq14: ata0 16200 116
Total 302136 2173

Over 1,110 interrupts in one second and it will just continue that
way forever..... Yet, if I then attach to the modem, all becomes OK
after that ?!?!

# cu -l /dev/cuau4
Connected
ati3
Manufacturer: Sierra Wireless, Inc.
Model: AC860
Revision: U1_2_40ACAP G:/WS/FW/U1_2_40ACAP/MSM6275/SRC 2006/09/11 17:50:27
IMEI: 357806001674042
IMEI SV: 17
FSN: X171696267512
3GPP Release 5
+GCAP: +CGSM,+FCLASS,+DS
2007/04/19
09:57:29


OK

# vmstat -i ; sleep 1 ; vmstat -i
interrupt total rate
irq0: clk 252979 995
irq4: uart0 718 2
irq8: rtc 32377 127
irq10: cbb0 uart4 225905 889
irq11: sis0 86 0
irq14: ata0 16392 64
Total 528457 2080
interrupt total rate
irq0: clk 254064 996
irq4: uart0 745 2
irq8: rtc 32516 127
irq10: cbb0 uart4 225905 885
irq11: sis0 86 0
irq14: ata0 16392 64
Total 529708 2077


I also confirmed that if I set hw.intr_storm_threshold=0 in the
/boot/loader.conf, the Soekris does indeed lockup after displaying
the uart4 on the console.


---Mike