World Wine News

"Our chairman has challenged the IT organisation, and indeed all of IBM to move to a Linux based desktop by the end of 2005," Greenberg wrote in a November 2003 memo. "This means replacing productivity, Web access, and viewing tools with open standards based equivalents," he said.

But doing it in an orderly fashion -- even assuming you could get buy-in from the individuals themselves, which in itself is a huge assumption -- is no overnight matter. And the fact IBM is still using Wine -- Windows emulation -- to run mission-critical applications such as Lotus Notes, its own product, is a demonstration of just how hard the process is.

Fun article ... author tests various viruses with WINE.

http://os.newsforge.com/article.pl?sid=05/01/25/1430222&from=rss

But under Linux if you mount a fat32 partition with all defaults then only root can write to it. If you want other users to have write to it, then you have to use the appropriate mount options to set the umask, user and group just right.

Yes, but then the kernel will only execute the file IF it has execute permissions - so when the worm drops BackOriface.exe on your drive and tries to run it, it won't as it won't have had the +x bit set.

And a worm smart enough to realize it is running under Wine and able to make the syscall to set the +x bit probably will be smart enough to get a native executable for the infection.

The biggest potential entrypoint for viruses in Wine is probably Internet Explorer. People who use that should know the risks involved, and it's massively unlikely anybody would use IE as their default browser on Linux.

I was going to wait a while (until it had a bit more content) before announcing this here, but as it seems people are setting up Wikis etc already here is my 2 cents:

http://navi.cx/svn/misc/trunk/winekb/index.xml

** THIS IS NOT A WIKI **

Instead, it's a mini website that uses a simple dialect of XML suitable for embedding code and random text into. It's designed to be:

• Easy to improve
• Visually appealing
• Controlled

What do I mean by these things?

Firstly, easy to improve: if you want to add some content to a page, save or wget the file to disk, edit it in your favourite text editor, and then send me the diff (or the full file and I'll diff it for you). You don't need any web design skill, as you can see if you do a "View Source" in the page.

Secondly, visually appealing: I dislike the way Wikis look most of the time. XSL:T is a nice language for doing tree transformations so I used it. The page code looks like this:

<kbpage title="Index">
<section title="Users">
* Installing <l>NativeMSI</l> and <l>NativeDCOM</l>
* The <l>ConfigFile</l>
* #winehq channel faq - read this when you have an issue
</section>

<section title="Developers">
* <l>AddingStuff</l> to this site
* <l>InterestingPatches</l>
* <l>WinelibAsBinaries</l>
</section>
</kbpage>

See? It's easy. Yet the resulting pages looks quite nice IMHO and are uniform in style. There's a new page template here:

http://navi.cx/svn/misc/trunk/winekb/template.xml

Thirdly, controlled: one of the objections Newman always had to a Wiki was that it'd fill up with crap or become another appdb, and then he'd have to maintain it. You can contribute to the knowledge base by sending patches to me or to wine-devel (if you think others would find the text useful), same as any other project. I won't be adding anything there that duplicates the appdb except maybe for extremely popular programs and libraries like MSI and DCOM.

You can hack on it in other ways: the code for the site is managed using Subversion/SVK, so you can check out a copy of it like so:

$ svn co http://navi.cx/svn/misc/trunk/winekb

If you want to do a lot of work on it, use SVK which will let you branch changes and commit locally so you can keep your patches separated nicely.

Right now there isn't much content. There's some stuff for users, like how to set up a basic config file, how to install native DCOM etc, and there is stuff for developers like a list of interesting-yet-bounced patches and some notes on the "Winelib programs as binaries" work me and Vincent did.

It's designed for content that doesn't really fit the main WineHQ website. If I think something makes more sense in the developer guide I'll ask you to send the content there.

In other words, the KB is designed to be half-way between the structured and professional WineHQ website and a Wiki.

Easy to improve

It is easy, but not as easy a regular Wiki. By a long shot. Moreover a Wiki like MoinMoin is so common on the web that people know it, and they don't have to learn yet another markup format. And you cannot compare the click-to-edit--edit--click-to-save ease of using a Wiki with the procedure to change this one (mentioning svk, diff, mail, etc, and you've lost 95% of people, right there).

Visually appealing

This is in the eye of the beholder. I am sorry, but MoinMoin doesn't look at all that bad that it's a reason to reinvent the wheel. Besides, a real Wiki has nice features (email notifications, history, etc) that we are all missing with a hand-rolled one.

Controlled

This has been a non-argument from the beginning. Jer has been put off by the silly FAQ-O-Matic, but that wasn't really a Wiki. There are so many Wikis on the web that work just fine and are useful (see the Arch Wiki, and so many others) that this has no support whatsoever. In fact, the lack of control (and the associated ease of editing) are the very reason to use Wikis, and we are giving these up for some irrational fears.

Let's add a Wiki to the main site (MoinMoin in particular), and see what happens. If it doesn't work out, we'll take it down. I'm sure people will love it.

Earlier I added hotel info to the WineConf pages: http://www.winehq.com/wineconf/travel

Here's what you need to know:

• we have 50 rooms reserved for Fri and Sat night, we have some on Thursday and Monday night too. I'll drink bier with anyone arriving early or staying late.
• 81 EUR for 2 nights
• you have to send mail to the address on the web page to reserve a room, note the "keyword"
• first come, first served, reservations must be made by March 29th
• did I mention RSVP'ing on WineHQ as well?

Thanks goes out to Hans-Ulrich Schmid (WRS) and Ulrich Gemkow (University of Stuttgart) for getting together and figuring this out.

Okay, we've got 19 rsvp's for WineConf. And that's missing me and a lot of other people that I know will be there, so shame on me and shame on you.

So get over to www.winehq.org/wineconf and rsvp (including fellow folks from CodeWeavers; if you're reading this, you better plan on being there <grin>).

In all seriousness, we really need an accurate head count so we can plan properly; if there really only will be 25-30 people, that makes things a good bit different than 75 people.

While you're there, join the wine-conf mailing list; I think that's the most appropriate place for us to start the discussions of the agenda and to start fine tuning the details. So if you don't subscribe to wine-conf, but you're planning to go, you're going to start missing out on everything but the nag messages <grin>

This document assumes you are familiar with the basics of DCOM. If you aren't read this first:

http://winehq.com/docs/winedev-guide/dcom-1

This is not suitable study material for beginners. Don't say I didn't warn you.

Apartments

Before a thread can use COM it must enter an apartment. Apartments are an abstraction of a COM objects thread safety level. There are many types of apartment but the only two we care about right now are single threaded apartments (STAs) and the multi-threaded apartment (MTA).

Any given process may contain at most one MTA and potentially many STAs.

This is because all objects in MTAs never care where they are invoked from and hence can all be treated the same. Since objects in STAs do care, they cannot be treated the same.

You enter an apartment by calling CoInitializeEx and passing the desired thread model in as a parameter. The default if you use the deprecated CoInitialize is a STA, and this is the most common type of apartment used in COM.

An object in the multi-threaded apartment may be accessed concurrently by multiple threads: eg, it's supposed to be entirely thread safe. It must also not care about thread-affinity, the object should react the same way no matter which thread is calling it.

An object inside a STA does not have to be thread safe, and all calls upon it should come from the same thread - the thread that entered the apartment in the first place.

The apartment system was originally designed to deal with the disparity between the Windows NT/C++ world in which threading was given a strong emphasis, and the Visual Basic world in which threading was barely supported and even if it had been fully supported most developers would not have used it. Visual Basic code is not truly multi-threaded, instead if you start a new thread you get an entirely new VM, with separate sets of global variables. Changes made in one thread do NOT reflect in another, which pretty much violates the expected semantics of multi- threading entirely but this is Visual Basic, so what did you expect? If you access a VB object concurrently from multiple threads, behind the scenes each VM runs in a STA and the calls are marshaled between the threads using DCOM.

In the Windows 2000 release of COM, several new types of apartment were added, the most important of which are RTAs (the rental threaded apartment) in which concurrent access are serialised by COM using an apartment-wide lock but thread affinity is not guaranteed.

Do we have any details about how you use RTAs? I have a feeling this was something that was due to be implemented for Windows 2000, but was dropped before it shipped.

Structure of a marshaled interface pointer

When an interface is marshaled using CoMarshalInterface, the result is a serialized OBJREF structure. An OBJREF actually contains a union, but we'll be assuming the variant that embeds a STDOBJREF here which is what's used by the system provided standard marshaling. A STDOBJREF (standard object reference) consists of the magic signature 'MEOW', then some flags, then the IID of the marshaled interface. Quite what MEOW stands for is a mystery, but it's definitely not "Microsoft Extended Object Wire". Next comes the STDOBJREF flags, identified by their SORF_ prefix. Most of these are reserved, and their purpose (if any) is unknown, but a few are defined.

After the SORF flags comes a count of the references represented by this marshaled interface. Typically this will 1 in the case of a normal marshal, but may be 5 for table-strong marshals and 0 for table-weak marshals (the difference between these is explained below).

This is wrong. It is 5 for normal marshals, and 0 for both table-weak and table-strong.

The reasoning is this: In the general case, we want to know exactly when an object is unmarshaled and released, so we can accurately control the lifetime of the stub object. This is what happens when cPublicRefs is zero. However, in many cases, we only want to unmarshal an object once. Therefore, if we strengthen the rules to say when marshaling that we will only unmarshal once, then we no longer have to know when it is unmarshaled. Therefore, we can give out an arbitrary number of references when marshaling and basically say "don't call me, except when you die."

The most interesting part of a STDOBJREF is the OXID, OID, IPID triple. This triple identifies any given marshaled interface pointer in the network. OXIDs are apartment identifiers, and are supposed to be unique network-wide. How this is guaranteed is currently unknown: the original algorithm Windows used was something like the current UNIX time and a local counter.

OXIDs are generated and registered with the OXID resolver by performing local RPCs to the RPC subsystem (rpcss.exe). In a fully security-patched Windows system they appear to be randomly generated. This registration is done using the ILocalOxidResolver interface, however the exact structure of this interface is currently unknown.

OIDs are object identifiers, and identify a stub manager. The stub manager manages interface stubs. For each exported COM object there are multiple interfaces and therefore multiple interface stubs (IRpcStubBuffer implementations). OIDs are apartment scoped. Each ifstub is identified by an IPID, which identifies a marshaled interface pointer. IPIDs are apartment scoped.

Unmarshaling one of these streams therefore means setting up a connection to the object exporter (the apartment holding the marshaled interface pointer) and being able to send RPCs to the right ifstub. Each apartment has its own RPC endpoint and calls can be routed to the correct interface pointer by embedding the IPID into the call using RpcBindingSetObject. IRemUnknown, discussed below, uses a reserved IPID.

Only in the implementation I've written and am currently testing now. The native version generates an IPID as per any other object and simply notifies the SCM of this IPID.

Both standard and handler marshaled OBJREFs contains an OXID resolver endpoint which is an RPC string binding in a DUALSTRINGARRAY. This is necessary because an OXID alone is not enough to contact the host, as it doesn't contain any network address data. Instead, the combination of the remote OXID resolver RPC endpoint and the OXID itself are passed to the local OXID resolver. It then returns the apartment string binding.

This step is an optimisation: technically the OBJREF itself could contain the string binding of the apartment endpoint and the OXID resolver could be bypassed, but by using this DCOM can optimise out a server round-trip by having the local OXID resolver cache the query results. The OXID resolver is a service in the RPC subsystem (rpcss.exe) which implements a raw (non object-oriented) RPC interface called IOXIDResolver. Despite the identical naming convention this is not a COM interface.

Unmarshaling an interface pointer stream therefore consists of reading the OXID, OID and IPID from the STDOBJREF, then reading one or more RPC string bindings for the remote OXID resolver. Then RpcBindingFromStringBinding is used to convert this remote string binding into an RPC binding handle which can be passed to the local IOXIDResolver::ResolveOxid implementation along with the OXID. The local OXID resolver consults its list of same-machine OXIDs, then its cache of remote OXIDs, and if not found does an RPC to the remote OXID resolver using the binding handle passed in earlier. The result of the query is stored for future reference in the cache, and finally the unmarshaling application gets back the apartment string binding, the IPID of that apartments IRemUnknown implementation, and a security hint (let's ignore this for now).

Once the remote apartments string binding has been located the unmarshalling process constructs an RPC Channel Buffer implementation with the connection handle and the IPID of the needed interface, loads and constructs the IRpcProxyBuffer implementation for that IID and connects it to the channel. Finally the proxy is passed back to the application.

Handling IUnknown

There are some subtleties here with respect to IUnknown. IUnknown itself is never marshaled directly: instead a version of it optimised for network usage is used. IRemUnknown is similar in concept to IUnknown except that it allows you to add and release arbitrary numbers of references at once, and it also allows you to query for multiple interfaces at once.

IRemUnknown is used for lifecycle management, and for marshaling new interfaces on an object back to the client. Its definition can be seen in dcom.idl - basically the IRemUnknown::RemQueryInterface method takes an IPID and a list of IIDs, then returns STDOBJREFs of each new marshaled interface pointer.

There is one IRemUnknown implementation per apartment, not per stub manager as you might expect. This is OK because IPIDs are apartment not object scoped.

In fact, according to the DCOM draft spec, they are machine-scoped, but this implies apartment-scoped.

Table marshaling

Normally once you have unmarshaled a marshaled interface pointer that stream is dead, you can't unmarshal it again. Sometimes this isn't what you want. In this case, table marshaling can be used. There are two types: strong and weak. In table-strong marshaling, selected by a specific flag to CoMarshalInterface, a stream can be unmarshaled as many times as you like. Even if all the proxies are released, the marshaled object reference is still valid. Effectively the stream itself holds a ref on the object. To release the object entirely so its server can shut down, you must use CoReleaseMarshalData on the stream.

In table-weak marshaling the stream can be unmarshaled many times, however the stream does not hold a ref. If you unmarshal the stream twice, once those two proxies have been released remote object will also be released. Attempting to unmarshal the stream at this point will yield CO_E_DISCONNECTED.

RPC dispatch

Exactly how RPC dispatch occurs depends on whether the exported object is in a STA or the MTA. If it's in the MTA then all is simple: the RPC dispatch thread can temporarily enter the MTA, perform the remote call, and then leave it again. If it's in a STA things get more complex, because of the requirement that only one thread can ever access the object.

Instead, when entering a STA a hidden window is created implicitly by COM, and the user must manually pump the message loop in order to service incoming RPCs. The RPC dispatch thread performs the context switch into the STA by sending a message to the apartments window, which then proceeds to invoke the remote call in the right thread.

RPC dispatch threads are pooled by the RPC runtime. When an incoming RPC needs to be serviced, a thread is pulled from the pool and invokes the call. The main RPC thread then goes back to listening for new calls. It's quite likely for objects in the MTA to therefore be servicing more than one call at once.

Message filtering and re-entrancy

When an outgoing call is made from a STA, it's possible that the remote server will re-enter the client, for instance to perform a callback. Because of this potential re-entrancy, when waiting for the reply to an RPC made inside a STA, COM will pump the message loop. That's because while this thread is blocked, the incoming callback will be dispatched by a thread from the RPC dispatch pool, so it must be processing messages.

While COM is pumping the message loop, all incoming messages from the operating system are filtered through one or more message filters. These filters are themselves COM objects which can choose to discard, hold or forward window messages. The default message filter drops all input messages and forwards the rest. This is so that if the user chooses a menu option which triggers an RPC, they then cannot choose that menu option *again* and restart the function from the beginning. That type of unexpected re-entrancy is extremely difficult to debug, so it's disallowed.

Unfortunately other window messages are allowed through, meaning that it's possible your UI will be required to repaint itself during an outgoing RPC. This makes programming with STAs more complex than it may appear, as you must be prepared to run all kinds of code any time an outgoing call is made. In turn this breaks the idea that COM should abstract object location from the programmer, because an object that was originally free-threaded and is then run from a STA could trigger new and untested codepaths in a program.

Oh well, it was nice in theory.

With the spring comes WineConf 2005, and perhaps some interesting presentations by people. Last time I went to WineConf I was asked to write a mini presentation to show newbies what Wine hacking was all about.

In the end, there weren't many non-Wine developers there (which is what you'd expect, really) so we just had a discussion about how to reduce the learning curve instead. In retrospect that was far more interesting and useful than any presentation I could have done.

Nonetheless, that presentation was there and when I wrote it, I also wrote a little XSL transform for slide shows. That's because I don't like PowerPoint that much, especially I don't like how sucky its web export is and slideshows are good to put on the web.

So here is the presentation itself: http://bylands.dur.ac.uk/~mh/wineconf2004/presentation.xml

Just load it into a web browser like Mozilla or even IE6 (konq doesn't support xslt, sorry). As you can see it's nicely Wine themed :)

Here is the transform if you want to use the same presentation system yourself: http://navi.cx/~mike/xslt-slides/gen.xsl

Just look at the source code of presentation.xml to see how to use it.

I'm hoping we'll have a similar mix for this years WineConf as last year - Lionels OpenGL/games presentation in particular was interesting, but I think the mix of discussion and presentations was good.

One problem with presentations is that they tend to be in quite specialist areas, eg a DCOM presentation wouldn't be all that useful because there aren't many people working on it. So I think a bias more towards discussion of process and project organisation, things like that, would be more useful.

That's just my humble opinion of course ;)