WineHQ

World Wine News

All the news that fits, we print.

08/10/2006
by Brian Vincent
Issue: 319

XML source
More Issues...

This is the 319th issue of the Wine Weekly News publication. Its main goal is to procrastinate on purchasing tickets to WineConf. It also serves to inform you of what's going on around Wine. Wine is an open source implementation of the Windows API on top of X and Unix. Think of it as a Windows compatibility layer. Wine does not require Microsoft Windows, as it is a completely alternative implementation consisting of 100% Microsoft-free code, but it can optionally use native system DLLs if they are available. You can find more info at www.winehq.org


This week, 405 posts consumed 720 K. There were 64 different contributors. 63 (98%) posted more than once. 38 (59%) posted last week too.

The top 5 posters of the week were:

  1. 54 posts in 53K by julliard at winehq.org (Alexandre Julliard)
  2. 36 posts in 43K by dank at kegel.com (Dan Kegel)
  3. 18 posts in 49K by wine-devel at kievinfo.com (Vitaliy Margolen)
  4. 14 posts in 12K by Andrew.Talbot at talbotville.com (Andrew Talbot)
  5. 14 posts in 12K by the3dfxdude at gmail.com (Jesse Allen)

News: 0.9.17, 0.9.18, CrossOver Mac Archive
News

This issue of WWN is way overdue. Expect the next few issues to be that way. I suspect things are going to be a bit busy around the editorial headquarters. This issue is horribly incomplete, but I decided to kick it out the door tonight otherwise it likely wouldn't be out before next week. Development has been steamrolling ahead. I'm pretty sure we've seen more patches this summer than any other. We've also had patches rolling in from lots of new developers. (By the way, welcome aboard!)

Since the last issue we've had two beta releases of Wine. The first, 0.9.17, noted the following changes:

  • Still more work on Direct3D shaders.
  • Now using the Gecko engine directly for MSHTML.
  • Better support for apps switching to full screen mode.
  • Support for multiple joysticks.

The last one, 0.9.18, from July 27th noted:

  • Still more work on Direct3D.
  • A lot of MSI bug fixes and improvements.
  • More compatible memory management.
  • Several fixes for Win64 support.
  • Some performance improvements.

Given the new release schedule, we may see another release as soon as today.

CodeWeavers posted some frequently asked questions regarding their work on CrossOver Office for Macs. There's a nice screenshot of Microsoft Project running on OS X. From the FAQ:

What Applications Will it Run?

We intend for it to run all the applications that our current CrossOver Office product runs, with some new additions in the Microsoft 2003 version products. We also hope to offer support for a limited number of games. However, the final mix of supported applications is still being determined.

Current plans show pricing to be $59.95 for a single copy.

While not strictly related to Wine, there was news several weeks ago of a native Linux port of Lotus Notes. It will be a Notes v7 client and is expected to ship on July 24th. Lotus Notes has remained one of the top requested applications for a native port for quite a long time. It's worked with Wine for several years and CodeWeavers supports earlier versions with CrossOver Office.

Finally, there was some news on DesktopLinux.com about running World of Warcraft with Wine .


WineConf 2006! Archive
WineConf 2006

We haven't said a lot about WineConf, but it's planned, and it's going to happen. Have you booked tickets yet? We're just about a month away. Everyone is invited, but the event is geared predominately at developers. If you're involved with another open source project, you might find it interesting to attend. Rob Shearman sent some details to wine-devel last week about it:

Our annual Wine conference is taking place in approximately 6 weeks time. It is an opportunity for you to put names to faces and to talk about projects that are too crazy to think about discussing on wine-devel! It is also an opportunity to share knowledge and to share some drinks!

When is it? 16th & 17th September 2006

Where is it? University of Reading

Who Should Come? Anyone interested in Wine development. Please note: this is a developers-only conference.

How To Sign Up? Check out http://www.winehq.org/wineconf for details. Please RSVP if you intend to come.

What's the Agenda? We'll be starting a thread on that shortly.

What's Urgent? The University is requiring that we have an exact head count for rooms on campus. What's more, we have to give them that count a month in advance, and they will not give us a refund for folks that don't show up or want to change things around. Hence, if you would like one of the convenient, reasonably priced rooms on campus, you need to follow the payment step at: http://www.winehq.org/wineconf/travel

as soon as possible to book your room. Note that since we won't get any refunds for cancellations, we won't be able to refund your money either :-( .

I thought WineConf 2005 was a lot of fun and I think everyone else who attended would agree with me, so hurry up and make your travel arrangements and RSVP now!

Jeremy White reiterated the importance of booking the hotel:

I know that WineConf is still over a month away, but you'll really regret it if you don't make your accommodation selections now.

If you paypal (or even arrange to mail a check) to me *now*, you'll get to hang out with all the cool people, on campus, and learn what the "Drink-making facilities on each corridor" really means, all for the low low price of about 40 pounds (okay, so the exchange rate sucks, and that's $75 US, but still...)

If you don't get me a room request in the next few days, then I'll have to close down our room reservation, and you'll be stuck in some sleazy motel on the edge of town paying outrageous sums of money. (Okay, they actually all look like nice places to stay, but they will be a ways away, and they all cost more :-/).

Again, see this page for more details:

If you can't do PayPal, I can take a check in US currency as well.

One last tip: kayak.com . It's a great site for finding flights.


CreateRemoteThread & VirtualAllocEx Archive
Memory Management

CreateRemoteThread and VirtualAllocEx do some tricks with memory that make implementing them on Linux difficult. Those functions and few others are known as virtual memory functions. VirtualAllocEx can reserve as well as commit a series of memory pages within the virtual address space of another process. CreateRemoteThread can then start a new thread that begins execution within that address space of the other process. As Alexandre once said, it's fairly tricky to do correctly.

Thomas Kho came up with an implementation of those functions for Wine. There was quite a bit of debate over how he did it and it took about three revisions of the patch. In the last revision he described how it works:

Much thanks for the comments on yesterday's patch of a service thread implementation for remote process operations. This patch uses APCs to implement CreateRemoteThread/RtlCreateUserThread, VirtualAllocEx/NtAllocateVirtualMemory and VirtualQueryEx/NtQueryVirtualMemory.

APCs are queued per-thread, so I made minimal changes to add a per-process APC queue. Of course, this implementation suffers from the limitation that the remote process must go into an interruptible wait for the operation to succeed; an operation targeting a suspended process would block.

Stefan Siebert mentioned that VirtualQueryEx is necessary for Lotus Notes Software Diagnostics (nsd.exe). I successfully tested this function with a simple program that prints a process' memory map:

Lastly, I should mention that I have two pending patches to add conformance tests for remote process operations:

Asynchronous procedure calls seem to be one way to go to attack the problem. They sort of work like signals on Unix. They can be used for timers, notifications, error reporting, and such. When it comes to APCs in Windows, there's two kinds: kernel-mode and user-mode. Guess which one has a more privileged level of operation? (By the way, here's a good page describing APCs and some of the low-level stuff they do.) All that led Dan Kegel to comment:

Tommy's APC version of his VirtualAllocEx / CreateRemoteThread patch seems to be safe (since APCs only run a points where threads in well-written programs are not holding locks), is complete enough to make many apps happy, and is probably the best that can be done without a service thread or a real kernel change. I've encouraged him to submit it for real. Let's get this puppy committed!

Alexandre thought there were still big issues, Unfortunately there's no such guarantee, kernel APCs will run on every wait. Unless you meant user APCs which only run on alertable waits, but this means they would essentially never run. Like so much of the Windows API, APCs are a feature that could have been useful if it had been properly thought out...

So the issue here is that the kernel APCs could run at a point where a thread is holding a lock. It led Tommy to ask:

I'm not too clear why this is a problem. I thought that kernel APCs run on just interruptible waits, which excludes the wait on suspend and perhaps others.

Also, I was under the impression that only locks taken in the code path of the VM and thread creation functions had to be respected. If necessary, non-interruptible waits could/should be done while holding these locks (although I looked through the code and it doesn't look like any applicable code does a wait while holding a lock).

Dan thought the APC mechanism might still be usable:

The closest Alexandre has come to answering this is

but with APCs we at least handle one of his two objections (an APC definitely doesn't interrupt any critical section initialization). leaving

    "One problem is that many locks have to be acquired in a specific order to avoid deadlocks, and since you don't know which locks the thread is already holding you can't guarantee the order"

So all you have to do is identify all the locks that your APCs might need to acquire, and verify that they are always acquired in the same order by all possible code paths.

Alexandre replied:

Well, you are right that running APCs only on waits is a lot safer than interrupting the thread anywhere. Unfortunately, interrupting the thread is really what you want. In particular remote operations on suspended threads have to be supported, since manipulating a process from the debugger is probably the number one reason for this feature.

Still, doing that stuff in APCs is a step in the right direction, you just need to make sure you can safely run these APCs from the SIGUSR1 handler.

There was more discussion after that, but it doesn't look like the problem has been solved. The good news is, we may finally be a bit closer to getting a really tough class of functions implemented.


Metacity Fullscreen Issues Archive
Window Management

Ahhhh... window management. Getting the behavior of windowing to match between X and Windows is tough. It's not quite enough to just make it work, you also have to make it work on every flavor of window manager out there. Sometimes one window manager decides it's going to interpret messages a slightly different way than another window manager. We have some new "fullscreen" code in Wine and Vincent Povirk stumbled on a problem:

Some patches were committed to wine recently to make it use _NET_WM_STATE_FULLSCREEN for fullscreen apps. These patches appear to work in kwin but not metacity (I was using firefox as a test). In metacity, it appears wine recognizes that it needs to make a window fullscreen and send the appropriate message, but nothing happens (panels are still on top of firefox in fullscreen mode).

I've mentioned this in Wine's bugzilla #3312 , but I suspect it's probably caused by some quirk in metacity that wine triggers by doing some rather odd things itself. That's why I've CC'd the metacity-devel list (to which I and probably anyone on any wine mailing lists are not subscribed).

So, uh, metacity people, under what circumstances would metacity be likely to put a window that had sent the proper _NET_WM_STATE_ADD message for _NET_WM_STATE_FULLSCREEN behind panels?

That email triggered a flood of troubleshooting. A few of the Metacity developers spent a lot of time trying to figure out what Windows programs do when they go fullscreen since whatever Wine was interpreting that as wasn't working. Vincent explained:

Windows does not have a fullscreen mode. In Windows, windows that are Always on Top are on top of panels. Windows applications enter "fullscreen mode" by positioning themselves to take up the whole screen and being always on top (actually, now I'm not so sure they have to be always on top; I'm only basing that on my experience as a Windows user).

When the size of a window or the screen resolution changes, Wine marks windows whose client area covers the whole screen as fullscreen. This is done in the update_fullscreen_state function in this file.

Rob Adams replied, Metacity has some heuristics for fullscreening windows that don't fullscreen themselves. IIRC, it looks to see if the window hits all the screen edges and then pretends that it's fullscreen for stacking constraints, etc.

Havoc Pennington explained in more detail:

Look at src/window.c:recalc_window_features() for possible reasons metacity decided to disable fullscreenability.

In this case it looks pretty clear though - the firefox window isn't resizable, metacity disables fullscreen in that case unless the window size is equal to the screen size and the window is not decorated.

It looks like 1280x1024 could be your screen size, so my guess is that the window has a frame on it.

I don't really know why the !window->decorated is in the check in recalc_window_features but Elijah may well.

This would explain why wine fullscreening works sometimes, it probably works when the window is a) full screen size and b) also disables the window decorations.

Fullscreen mode would remove the decorations anyhow though, is why I don't know why metacity checks this.

There was then a lot of back and forth emails between Dmitry, Havoc, and Elijah Newren. Havoc explained how to generate some logs from Metacity and Dmitry was able to compare those against the ones generated from Wine. Havoc compared some of the logs and deduced the sequence of events leading up to the problem.

From the metacity log, it looks to me like Wine withdraws the window, turns on window decorations, then remaps the window.

Metacity then has to unmap/map one more time in order to place the window inside its window frame. Undecorated windows don't have a frame so don't have the extra unmap/map caused by reparenting, but normal windows do.

Clients and WMs both have to distinguish between maps that are incidental to implementation details and maps that have a meaning in the ICCCM. Essentially you need a "withdrawn" flag on windows indicating whether you have yourself mapped or unmapped the window. The WM may also map/unmap in order to reparent, manage workspaces, etc. and you should just ignore those map events most of the time.

If you need a "steady state" mapped status that matches the Windows API, then you would need to "simulate" it on the client side, by e.g. using the client-side withdrawn flag instead of the mapped state.

That led Dmitry to come up with some patches for Metacity to fix the problem:

I've added a patch which fixes the problem for me. Here is a comment I've put there to explain why the patch is needed:

    "The problem appears to not be related to the heuristics Metacity uses to check whether a window may be fullscreened, but rather to a missing update of a window state after screen resolution change so that heuristics can do their work.

I've used slightly old Metacity 2.10.0 sources, but hopefully the attached patch is still relevant."

Please let me know whether the patch is a right way to fix the bug, or you need further explanations.

Vincent tried out one of the patcches and reported it worked. However, there was an issue returning from full screen mode back to a window. Elijah explained the issue:

Actually, that's because Dmitry was using an older version. The patch actually applied to head would have also applied to 2.14.3. :) Anyway, the returning from fullscreen mode bug makes perfect sense, and I'm pretty sure I know the cause: metacity tries to return the window to the size it was before it was fullscreened, and it sounds like it was successful in this case. Remember that the application resized itself to the size of the screen, then you (wine) noticed that and decided to send the fullscreen toggle message. Thus, the size the window had before being fullscreen, happened to be a size equal to the size of the screen. Since wine has added heuristics for detecting when an app sets its size to the size of the screen, I think it either needs to add heuristics for resizing back to normal size, or to just have the fullscreen toggle message replace the apps' resizing commands instead of supplementing them.

Dmitry had a suggestion for fixing it as well.

So as it stands, it appears newer version of Metacity should support fullscreening within Wine.


Preloader Problem on x86_64 Archive
Fixes

Pavel Roskin ran into a strange problem on a bleeding edge system:

Something strange to happened my Wine installation recently. I'm getting the following messages if I run any program from Wine:

    $ winemine
    wine_main_preload_info not found
    err:dosmem:setup_dos_mem Cannot use first megabyte for DOS address space, please report
    wine_main_preload_info not found
    err:dosmem:setup_dos_mem Cannot use first megabyte for DOS address space, please report

The programs still work fine after those for lines are printed. What's worse is that that Windows PE files don't work at all, including those that come with Wine:

    $ wine winhlp32.exe
    wine_main_preload_info not found
    wine: could not load L"C:\\windows\\winhlp32.exe": Module not found

I'm using current Wine from git on Fedora Core development (future FC6) for x86_64, glibc-2.4.90, gcc-4.1.1, Linux 2.6.18-rc2 (actually, the current version from the wireless-2.6 branch). The CPU is Intel with EM64T. Wine is compiled with default settings for i386 by the i386 compatibility compiler and i386 compatibility libraries included in Fedora. I removed ~/.wine to start with a clean system.

Since I keep updating the kernel and Wine almost every day and I upgraded from FC5 to Development recently, it's hard to say what's the exact reason.

P.S. I have a 32-bit Ubuntu 6.04 installation on the same machine, so I used chroot to compile and run Wine under Ubuntu under the same 64-bit kernel. I also used linux32 to fool uname into thinking it's an i686 machine. The messages from ELF programs are gone, but PE files still fail:

    $ winemine
    $ wine winhlp32.exe
    wine: could not load L"C:\\windows\\winhlp32.exe": Module not found

Moreover, if I run this Wine on Fedora, the results are the same - no more messages about wine_main_preload_info and setup_dos_mem. I guess it's a compile issue. Unfortunately, the difference between config.status in both builds is quite significant to pinpoint the reason.

Mike McCormack thought the problem was with Wine's preloader:

Looks like the wine-preloader isn't being run, so memory that wine needs isn't reserved.

Maybe check that configure is setting the EXTRA_BINARIES variable?

    mike@black:~/wine$ grep ^EXTRA_BINARIES loader/Makefile
    EXTRA_BINARIES = wine-kthread wine-pthread wine-preloader

Pavel did more debugging and noticed something odd:

It is installed and it is run. If I replace wine-preloader with a script that logs the arguments and runs the renamed wine-preloader, I see it being called twice when an exe file is run with Wine. For example:

    $ wine FarManager170.exe
    wine_main_preload_info not found
    wine_main_preload_info not found
    wine: could not load L"Z:\\home\\proski\\dist\\windows\\FarManager170.exe": Invalid address

Mike saw a problem there related to the last line:

OK, it seems to be that the preloader can't lookup the ELF symbol "wine_main_preload_info" in wine-pthread.

That's bad, because the preloader is reserving memory, but has no way to let wine-pthread know which memory it reserved.

Can you see that symbol exported from wine-pthread?

    mike@black:~/wine$ nm loader/wine-pthread | grep wine_main_preload_info 7bf023d8 B wine_main_preload_info

Pavel agreed that was the problem. After digging around more he found a workaround:

find_symbol() looks for DT_HASH (4) to set symtabend and fails to find it. As I understand it, we should trick ld into creating the DT_HASH tag or use some other way to locate wine_main_preload_info.

The 6ffffef5 tag is DT_GNU_HASH - that may be a good alternative to DT_HASH. Adding -Wl,--hash-style=sysv to the gcc flags when linking wine-pthread solves the problem!!!

I think adding support for DT_GNU_HASH would be a good idea anyway.

Mike thought that was reasonable:

Great work! There's some information on DT_GNU_HASH here:

Looks like we don't actually use the hash table to look up the symbol, just to find the last entry in the symbol table.

I agree with you the best way to solve the problem would be to handle DT_GNU_HASH.


Windows Notes Client with Wine Archive
Fixes

As we mentioned in the news section above, there's going to be a Lotus Notes port to Linux. Well, there's still going to be some missing features in the near term, such as Designer not being included. There's also concern that the reliance on Java could easily require more resources (memory) than Wine and the Windows Notes client. Dan Kegel put together a nice summary on the wiki about all that. There's also been an effort lately by a few developers to get the Notes 7 client running. A few days ago Dan reported he got it running:

Thanks to a few targeted patches written by Stefan Siebert and James Hawkins, and of course decades of effort by the whole Wine team, Lotus Notes 7 trial version now installs and runs! See https://wiki.winehq.org/LotusNotes for details.

I just spent an hour or so playing with Notes as a plain old email client. It had a few mysterious crashes, but in general seemed usable. (Not that I'd ever use it after getting used to gmail, but that's another story.) I filed bug #5752 about a strangeness in the email client.

David Goodenough mentioned another way the Windows Notes client could be useful:

Good news. One possible "market" for this is in the Debian world as currently the Linux Notes client will not install as it cannot work out what version of Mozilla is installed. IBM have never really made any of their stuff easy to install on Debian. Of course the Debian sid wine will need to be brought up to date (currently .15) and will need the two patches mentioned above included.

Stefan Siebert pointed out how that problem might be able to be solved, It needs GRE (gecko runtime environment) and searches for gre.conf in /etc Have a look on http://developer.mozilla.org/en/docs/GRE for more information.


Winelib Port Archive
Winelib

Winelib hasn't gotten much attention lately. In fact, some of you reading this might not have any idea what Winelib even is. Besides just letting you run Windows programs, Wine has Winelib , which lets you link your program against Wine's own DLLs. That means anywhere you can run Wine you could recompile your Windows program and it would run. Of course, that's easier said than done and ports to platforms other than x86 receive little attention.

Life in the x86 world has gotten pretty interesting over the past year though. There's an effort going on to get Wine running on Mac OS X and another to get Wine running on native 64-bit OS. (Wine currently works fine with 32-bit compatibility libraries.) It led Luc-Eric Rousseau to ask some questions about Winelib. Dan Kegel provided answers, so we'll go with a question/answer style format on this thread:

Luc: is there a definitive list somewhere what API WinLib implements of Win32, and what it doesn't? I have had some difficulty finding this information.

Dan: Each of Wine's DLLs comes with a file which lists which APIs it implements, e.g. atl.spec The lines that say 'stub' are not really implemented.

http://www.winehq.org/winapi_stats has a summary of this data, I think. But even some of the ones that don't say 'stub' are only partial or sham implementations. The real way to tell how many bugs your Windows app will run into is to try simply installing the Windows app on Linux using Wine. Does your app have a regression test or GUI test framework of any sort? If so, you can run that under Wine and compare results.

Luc: one application we'd like to port uses active scripting (VBScript). Basically we wrote some features in VBScript, which in turn automate our application (like macros in Photoshop CS). What would be our options for this if we were to use WinLib? (MainWin ships with VBscript and JScript)

Dan: Wine doesn't currently support VBScript (unless you install WSH from Microsoft). That could be added, but it'd be a substantial effort.

Luc: Would the Ole Compound Storage format (StgCreateDocfile) WinLib read/write be binary compatible with the one on Windows?

Dan: Should be.

Luc: what are the options of an application that embeds the IE web browser control?

Dan: That's supported now, though you may find that Wine's implementation has rough edges. Nothing a little support contract with CodeWeavers couldn't iron out.

Dan added:

Incidentally, if you're sticking with x86 only, you should consider just shipping your Windows .exe unchanged, and using Wine instead of Winelib. It works just as well, and it's a lot easier. That's what was done for Picasa.

If you haven't already seen it, you might want to look at my Wine for Windows ISVs page, http://kegel.com/wine/isv

I expect you'll run into some issues when testing your Windows executable on Linux with Wine. When you do, I'd be glad to try to help resolve them (though I might need you to send me a copy of the software so I can reproduce the trouble locally).

There was an issue brought up about Windows Scripting Host and distributing it. Dan briefly looked at the license and thought it should be ok, but suggested getting a lawyer to read it.

There was also a plug given to CodeWeavers for their excellent port of Picasa using Wine.


All Kernel Cousin issues and summaries are copyright their original authors, and distributed under the terms of the
GNU General Public License, version 2.0.