Hi Dave,
thanks for your explanation. I find your statements always enlightening.
I have to add only one remark: having optional features in a standard is
ok. It makes sense, although, while designing the optional feature to
think about the implications on existing or forthcoming applications. If
the amount of work needed for implementing the option outweights the
benefit (either because of complexity or because there are other ways to
do almost the same), then putting the option into the standard is bad. It
is bad (and not neutral), because it makes the standard 'thicker' and
raises the energy needed by 'outside' people to understand/implement it.
This, again, is from my experince with telecommunication standards. Many
of them are 'optionalized' to the extend that two implementations, both
claiming conformance to the standard, may not interwork...
Thanks,
harti
On Fri, 1 Aug 2003, Dave Butenhof wrote:
DB>Harti Brandt wrote:
DB>
DB>>On Fri, 1 Aug 2003, Dave Butenhof wrote:
DB>>
DB>>DB>Harti Brandt wrote:
DB>>DB>
DB>>DB>>On Fri, 1 Aug 2003, Alexander Terekhov wrote:
DB>>DB>>
DB>>DB>>AT>Harti Brandt wrote:
DB>>DB>>AT>[...]
DB>>DB>>AT>> I wonder how one would implement a per-thread current directory
DB>>DB>>AT>> in an environment where there is no 1:1 mapping between user
DB>>DB>>AT>> threads and kernel threads.
DB>>DB>>AT>
DB>>DB>>AT>Something similar to (but probably a bit less convoluted ;-) )
DB>>DB>>AT>a per-thread signal mask, I guess.
DB>>DB>>AT>
DB>>DB>>AT>> While I could think of ways, they seem too convoluted to me.
DB>>DB>>
DB>>DB>>Given that I have only one per-process current working directory this
DB>>DB>>would force me to intercept all systemcalls that contain pathnames and
DB>>DB>>fiddle with these, doesn't it?
DB>>DB>>
DB>>DB>>
DB>>DB>Yes, very likely, if you insisted on doing this on a kernel that's not
DB>>DB>designed to support it. Of course you'd also need to synchronize your
DB>>DB>kernel threads to be sure that only one could hit such a point at any
DB>>DB>time since you can have only one process kernel state at a time.
DB>>DB>
DB>>DB>This doesn't really even have anything to do with whether you've got 1:1
DB>>DB>or M:N mapping of threads. If your kernel has a CWD per process and
DB>>DB>you're trying to make it per-thread, you're in trouble.
DB>>
DB>>That's just what I was trying to get. When Posix requires me to re-design
DB>>my Unix kernel to fit into Posix's thread model, something is wrong. With
DB>>Posix, not the kernel. If I have 1:1 mapping things may actually be easy,
DB>>because I can move the cwd from the per-process structure to the
DB>>per-process structure. If I have a M:N mapping, I must handle this in user
DB>>space and repeating half of namei() in the pthreads library is a clear
DB>>sign of wrong design (of the specification).
DB>>
DB>POSIX doesn't require that any implementation support M:N, and many
DB>don't. As an implementor, that's your decision. If you aren't willing to
DB>do the work to support M:N, then presumably your decision is made, and
DB>that's fine. If you prefer not to bother but your customers insist,
DB>well, that's tough, but you can't blame it on POSIX. ;-)
DB>
DB>But of course POSIX conformance does require that your kernel be
DB>[re]designed to implement the standard. But there's a critical semantic
DB>distinction here: "POSIX" doesn't require anything; only if you CHOOSE
DB>to CONFORM to POSIX do you place requirements on your implementation.
DB>
DB>Given POSIX history, a new per-thread CWD would be an OPTION in POSIX
DB>and XSI, and you would choose whether to implement that option. This is
DB>indeed the principle justification for the confusing and sometimes
DB>annoying set of optional features -- because supporting them can often
DB>require substantial development investment, and you shouldn't be
DB>required to do that if it's not important to you and your customers.
DB>Even _POSIX_THREADS is a POSIX option that's not required for POSIX (or
DB>even POSIX 1003.1-2003) conformance. The Open Group chose to make this
DB>particular option mandatory for UNIX branding for UNIX 98 and beyond,
DB>but even so you're not required to seek that branding.
DB>
DB>However, for anyone who DOES implement an option, POSIX provides a
DB>common syntax and (within limits) semantics for that option so that
DB>portable programs can exploit the feature where it exists.
DB>
DB>>DB>But there's nothing unique about this. If you want to have multiple
DB>>DB>threads within a process then each needs its own registers, including
DB>>DB>stack and instruction pointers. In POSIX, each has its own signal mask.
DB>>DB>If you want each to have its own "security persona", you need to store
DB>>DB>that. If each has its own CWD, you need to store that, too. To do M:N
DB>>DB>you either need to explicitly context switch each piece of data
DB>>DB>(expensive when they're kernel state), or you need abstracted
DB>>DB>user-managed contexts. In our case, the kernel gets "all this stuff"
DB>>DB>through the thread register. Adding a per-thread CWD would be a trivial
DB>>DB>matter of adding the field to the "shared area".
DB>>
DB>>So that comes down to the question that I saw earlier in this thread:
DB>>if a thread has everyting a process has, what's the difference between
DB>>them? I cannot pin it down, but I have the feeling that if you find
DB>>yourself re-implementing the kernel in libpthreads there is something
DB>>with the model. If you find that you can globally rename process -> thread
DB>>in the kernel, something is wrong with the model. Anywhere there is a fine
DB>>border between things a thread should have and what it should not have...
DB>>
DB>There's always a fine line. Yes, calling a "process" a "thread" doesn't
DB>enable anyone to do something they couldn't do before. It's easy to
DB>label and characterize the extremes; it's the middle ground that gets fuzzy.
DB>
DB>There were many battles fought to draw the line that's currently in the
DB>standard. Many of them were fought multiple times, often with slightly
DB>different casts. What's there is the result of often difficult
DB>compromises. The signal model is a good example, because several signal
DB>models were proposed, developed, written up, and completely trashed
DB>(over many badly flame-singed heads) before Nawaf Bitar finally came up
DB>with the revolutionary "grand unified signal compromise" that remains
DB>largely intact in the current standard. Not because it's perfect, but
DB>because it was something everyone could understand, implement, use, and
DB>live with. (And I'm sure the fact that everyone in the working group at
DB>the time was sick and tired of the battle didn't hurt.)
DB>
DB>Fork was another example, because it's almost a fundamental and
DB>universal contradiction with a shared address space -- yet so deeply
DB>ingrained in POSIX/UNIX culture that it couldn't possibly be fixed or
DB>replaced. The posix_spawn function was a long time coming, and probably
DB>will never be widely used because it came so late.
DB>
DB>And yet, making CWD per-thread doesn't require "re-implementing the
DB>kernel in libpthread". It simply requires cooperation. Libpthread isn't
DB>just an add-on library; it's a part of the system, just like libc.
DB>Inseparable and deeply intertwined. An extension of the kernel into user
DB>space, really. And in a 1:1 implementation it need be no more than a set
DB>of syscall stubs. Although even on a 1:1 implementation it usually IS a
DB>lot more, because many operations can be optimized knowing you're
DB>already in user space, without crossing the protection boundary into the
DB>kernel. You can optimize mutexes by taking uncontended locks in
DB>user-space, for example. You might choose to optimize chdir() by putting
DB>the CWD into user space and avoiding a kernel call; but nobody requires
DB>that. If you're building a kernel to support M:N schedulers, then you
DB>probably want to do something like our "shared area", so the whole block
DB>of thread state (registers, CWD, signal mask, and whatever else) gets
DB>"switched" simply by the act of changing the processor's "thread
DB>register" (or various equivalents). But the standard, again, doesn't
DB>require any of this.
DB>
DB>>Methinks that the CWD is and should stay a per-process attribute, as is
DB>>the root directory.
DB>>
DB>>
DB>That was of course the decision of the working group at the time! But
DB>not all decisions are permanent. I do think that we ought to be a bit
DB>more conservative about radical changes now, though. In the original
DB>working group we could throw out a signal model and start from scratch
DB>because there were no implementations. (Even though DCE threads existed,
DB>the standard said "thou shalt not implement", so there WERE no
DB>implementations! ;-) ) Now, such a change would be a radical and
DB>unreasonable burden on too many people, for too little benefit to
DB>anyone. But THAT is an opinion, not a law...
DB>
DB>
--
harti brandt,
http://www.fokus.fraunhofer.de/research/cc/cats/employees/hartmut.brandt/private
yyyyyy@xxxxxxxxxxxxxxxxxxx, yyyyy@xxxxxxxxxxx
|