Writing a plain text file to disk from kernel space

[Lawrence Stewart]

Hi all,

I'm in the process of writing a kernel module to instrument some parts
of the network stack in FreeBSD 6.2. Everything works perfectly, except
that writing the logging info to syslog using printf is too inefficient
for this particular task. Given that log messages need to be generated
for each packet entering/leaving the network stack, my testing thus far
has showed that syslog groans under the load and uses far too much CPU time.

So, I'm looking for a way to manually open up a file from within kernel
space and dump characters into it.

I've come across the following threads that are related to my problem:

http://lists.freebsd.org/pipermail/freebsd-hackers/2007-May/020330.html

http://lists.freebsd.org/pipermail/freebsd-fs/2005-September/001306.html

Neither thread has provided the "ah-ha" information/discovery I'm
needing to get this working, and so I'd be very grateful to anyone that
can shed some light on how to piece this together in the most efficient
way possible.

To summarise the things I've experimented with thus far:

1. Following the "open" syscall down into kernel land and investigating
how the underlying kernel functions work. The "kern_open" function in
"kern/vfs_syscalls.c" seems interesting.

However, all of interesting functions that I've found take a thread
struct pointer as the first argument. I have not been able to figure out
where to get a thread struct pointer from. So perhaps if someone can
shed light on how to go about creating a thread struct suitable for
passing to these functions, I may be able to figure it out. I've tried
creating a new kthread using kthread_create, and got this working, but
this still didn't give me access to a thread struct pointer. I've also
tried accessing the thread queue for process 0 but didn't have any luck
there either (though I could have been accessing it incorrectly!).

2. Following the kernel panic code where it writes a dump out to disk.
This uses very scary low level structs and functions that I couldn't
wrap my head around... appeared to be below the vfs layer in the kernel,
so I suspect its to far below where I want to be at.

3. I followed the ktrace code, but that seemed to lead to a dead end of
passing logging data up to userspace via the ddb subsystem, which is not
what I want to do.

4. I followed coredump code, and this looked very interesting. It
operates directly on vnodes, but once again, the elusive thread struct
pointer is needed.



The "kern/vfs_*" files seem to contain the functions relating to all the
action of creating/writing/reading files (vnodes)... is there a good
document/tutorial somewhere that describes the layering and various
abstractions in the FreeBSD kernel fileystem related code? TCP
Illustrated has been invaluable in describing the layout and structure
of the various network stack components/functions/structs in the kernel.
Something similar for filesystem related tasks would be most useful.

I suspect that once I understand the mechanics of where thread struct
pointers come from and how to get my hands one one from within the
kernel, I'll be able to get the rest going.

Many thanks for your time and help.

Regards,
Lawrence Stewart

[Marc Lörner]

Hello Lawrence,

in file kern/vfs_syscalls.c you can see the kern_open function that uses
vn_open to open a file/vnode.

There are more vn_* functions like vn_rdwr for reading and writing to a vnode
or vn_close to close a vnode. To see how theses functions work, I suggest you
grep for them in the kernel-files.

BTW, you can get an thread-struct pointer from curthread (c.f. sys/pcpu.h).

HTH,
Marc Lörner

[Ivan Voras]

Post by Lawrence Stewart
I'll have a play around and report back to the list what I find for
archival purposes.

Please do, and also consider writing a short and instructive tutorial on
it! Many people have asked this same question without a to-the-point answer.

[Dag-Erling Smørgrav]

Post by Lawrence Stewart
I'm in the process of writing a kernel module to instrument some parts
of the network stack in FreeBSD 6.2. Everything works perfectly,
except that writing the logging info to syslog using printf is too
inefficient for this particular task. Given that log messages need to
be generated for each packet entering/leaving the network stack, my
testing thus far has showed that syslog groans under the load and uses
far too much CPU time.
So, I'm looking for a way to manually open up a file from within
kernel space and dump characters into it.

Look at the ktrace code.

Note that it opens the file in userland and passes it down to the
kernel. You may want to consider a similar mechanism.

DES

[Joseph Koshy]

ls> So, I'm looking for a way to manually open up a file from within
ls> kernel space and dump characters into it.

des> Note that it opens the file in userland and passes it down to the
des> kernel. You may want to consider a similar mechanism.

hwpmc(4) takes a similar approach, using a dedicated kthread to
to perform writes.

[Lawrence Stewart]

Hi Marc,


Marc Lörner wrote:

[snip]

Post by Marc Lörner
BTW, you can get an thread-struct pointer from curthread (c.f.

sys/pcpu.h).
[snip]


Thanks very much for the reply.

"curthread" is the missing key that I've been looking for! Thank you so
much. I've been banging my head against a brick wall for days now.

I'll have a play around and report back to the list what I find for
archival purposes.

Cheers,
Lawrence

PS Just found a thread I hadn't found before
(http://lists.freebsd.org/pipermail/freebsd-hackers/2006-March/015647.html)
which has lots of useful info and a link
(http://people.freebsd.org/~pjd/misc/kernio/) to some sample code!

[Dag-Erling Smørgrav]

I suspect that you can't use a file descriptor that was opened in one
thread in a completely different thread, but I'm not sure if this is
true, and if it is true, how to get around it.

A file descriptor is an index into a file table. Different threads have
different file tables.

If you want to read from or write to files within the kernel, you need
to operate directly on vnodes, not on file descriptors.

DES