Socks proxy gets error and won't start

[saubrey]

I upgraded to 6.1 from 6.04. Socks proxy gets an error and wont' start. Error message says "11/06/05 22:41:15 Error: Service failed startup on interface 192.168.0.2:1080 error 10013". Not sure what the problem is. It worked correctly with 6.04. MS TCP Loopback and Local Area Connection both show an error in the Socks Bindings tab. How do I fix? Thanks, Steve

[adrien]

OK, that's very interesting.

I saw that on one of the servers here the other day, and assumed it was a port conflict, but 10013 is WSAEACCES which means access denied.

If you stop then start the SOCKS service, does it go away? Is there anything else showing as running on port 1080 if you run "Netstat -an" from the command line?

Adrien

[saubrey]

The ntfrs.exe process (File Replication Service used by Active Directory) was using TCP port 1080. I stopped, then restarted the File Replication service and this time it is using port 396. WG's Socks Proxy now starts correctly.

[saubrey]

OK, now I'm confused. I rebooted the win2k server running WG and now DNS.EXE is listening on 1080 and WG's socks proxy failed to start. I have not made any changes to my Win2k server, but now after upgrading to WG 6.1 I'm having problems with win2k services using 1080. In the three previous years of running win2k and WG I've never had this problem. Do you have any ideas what is happening?

[adrien]

I would be reaching for the AV scanning software....

Adrien

[saubrey]

KAV found no viruses. It did find 12 programs that it said contained psKill, but it said that they weren't viruses. It seems that the Win2k & Active Directory services are using ports 1025 - 1099 and just by chance the DNS server or ntfrs service, etc. is using 1080. I'm just guessing at this...does this seem normal? Thanks for your advice. -- Steve

[adrien]

any port from 1024 -> is deemed ephemeral, and is fair game for any service to bind to.

Not sure why those services started binding to those ports.

Might just be easiest to change the SOCKS server port number. Would require changing the clients out.

Otherwise you would need to get WinGate to start before those services.

Adrien

[saubrey]

I uninstalled 6.1 and re-installed 6.04 and all of the problems I was having went away. Not sure why, but now everything is once again good. Socks proxy now starts because it seems that Win2k services skip using 1080. They use 1078, 1079, 1081, but skip 1080...not sure why. After installing 6.1 my client computers started to randomly not be able to access the printer installed on the server (same server as WG), but now with 6.04 printing is working again. With 6.1 I was unable to access the web sites on my IIS web server (same server as WG), but now with 6.04 I can.

Prior to re-installing 6.04 I did another AV scan using TrendMicro and it also found no viruses.

[adrien]

That's really preplexing.

We didn't change anything to do with service startup, or ports or anything like that with the release of 6.1.0, so I wonder whether MS looks for WinGate somehow to know whether to avoid those ports or something???

Also, running IIS on the same server - this should be no problem, how were your clients connecting to this server? Still through the proxy, or directly?

As for the printer...?

I'll take another look at the history of the code changes....

Adrien

[saubrey]

Another problem I had when 6.1 was installed is that psinfo \\server2 could not access \\server2. It got "Can't access server2" or some such error. Prior to installing 6.1, psinfo \\server2 has worked correctly for many years. The problem I had accessing the IIS web site was from over the Internet, so no it wasn't via the WG www proxy. I would get an "Internal server error 500" displayed in the browser when attempting access web sites. I never had this problem with 6.0.4.

Could the WG 6.1 firewall have been blocking access to the server? Just a thought, I don't really know what was happening. I checked the logs, but no blocks were shown. The firewall would explain losing access to the printer and to the IIS web sites. Although I don't think it explains port 1080 being used by Win2k services preventing the Socks service from starting.

This weekend I'll try re-installing 6.1 again. Maybe there was some glitch that I didn't notice when I installed 6.1 the first time.

[adrien]

Hi

Did you have any service in WinGate running on port 80?

I'm reading some docs that indicate that MS changed the way they handle multiple apps trying to listen on the same port, with a SO_REUSEADDR socket option.

I didn't think we changed that at all, but it seems to be related.

So I'm wondering if IIS was having trouble starting up - I'd take a look in the event log.

Adrien

[adrien]

Looks like you can reserve ports in the registry.

will have to check installer code to see if we had this in but removed or something.

http://support.microsoft.com/default.aspx?scid=kb;en-us;812873

shows you how you can stop the OS from allocating ephemeral ports in a certain range.

Probably wouldn't help with that IIS issue though.

PS, error 500 Internal server error means the connection made it in, and was received by a socket in the OS, therefore the firewall wasn't blocking it.

Adrien

[saubrey]

IIS is listening on port 8080. WG's www proxy is running on port 80. WG's Port Security redirects Internet requests for port 80 to port 8080. The Event Log does not show that IIS had problems starting. The Event logs does show a few other errors that only occurred when WG 6.1 was installed:
1. DNS Server Event ID 4000. The DNS server was unable to open Active Directory. This DNS server is configured to obtain and use information from the directory for this zone and is unable to load the zone without it. Check that the Active Directory is functioning properly and reload the zone. The event data is the error code.
2. DhcpServer Event ID 1051. The DHCP/BINL service has determined that it is not authorized to service clients on this network for the Windows domain: mydomain.com.

That's all I could find out of the ordinary in the event log.

[saubrey]

I checked the Registry. ReservedPorts is not specified. So WG 6.04 is not setting this entry.

Could it be that WG is booting up before the other Win2k/Active Directory services and WG is getting port 1080 before another Win2k service can get it?

[saubrey]

I don't know if this is helpful or not. I have turned off the following services in WG: DHCP service, Winsock Redirector Service, DNS Server, & SMTP Server. These services have been off for two years. I use Win2k's DNS and DHCP services. Could it be that WG 6.1 did not correctly turn off these services and they conflicted with Win2k's? Which then caused Active Directory to screw up, which then caused all my problems?

[adrien]

Hi

You should be able to tell if WinGate is listening on those ports with netstat, or our PortList utility (part of NetPatrol).

I think what is happening is the AD services are starting before WinGate, and that's why WinGate is unable to start the SOCKS service.

Those DNS and DHCP system errors are worrying though.

We are seeing the SOCKS issue on one of our servers here (not normally running WinGate), so we have an environment to debug in.

Adrien

[adrien]

Hi steve

I've been doing some more testing on our 2k AS machine here that was showing this problem.

tried an engine that didn't set SO_REUSEADDR on the sockets - that didn't help, and in fact then the WWW proxy wouldn't start either, so I know that option is working (albeit not for SOCKS).

Then tried the WinGate 6.0.4 exes on that same machine. Same problem.

So that means it's not the new wingate executables. That only leaves something about the installs, or registry.

Adrien

[adrien]

With more testing am getting inconsistent results.

Uninstalled WinGate 6.1.0 and installed 6.0.4. On reboot, SOCKS server was running fine.

Swapped out exe's with 6.1.0 ones, rebooted, SOCKS server wouldn't start.

Swapped back out to 6.0.4 ones, rebooted, again, wouldn't start.

So, I'm wondering if it's a timing related issue (start order etc). In any case it isn't consistent, and therefore the only way I can think to make sure you will get the services starting you want every time is by forcing RPC to use specified ports, or reserving some ports.

Adrien

[adrien]

I tested with reserving a port.

You need to use regedt32.exe instead of regedit.exe since you need to create a REG_MULTI_SZ value called ReservedPorts under

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\TCPIP\Parameters

need to reserve the range, so I added "1080-1080" and a blank line after.

Need to reboot after that.

Works a treat though. We put it into the installer, so next release will do this for us.

Adrien

[saubrey]

The Registry Reserver ports trick seems to work good. I reserved port 1080, re-installed 6.1, and Socks started up correctly...yeah!

However, 6.1 only ran for a little over 24 hours and I had to unsintall it and go back to 6.04 again. 6.1 started behaving flaky like it did last week. Everything was running fine for the first 24 hours or so and then all of a sudden my IIS server could not be accessed from the Internet...all requests to it timed out. My POP3 email server (I don't use WG's email server) also could not be accessed from the Internet. About the same time internal client computers could not get Internet access (all client computers use the WG's www proxy manually configured in IE) Gatekeeper when launched on the WG server did not start up correctly...the outline of the GateKeeper window displayed with the outline of the Control panel in the left pane and the outline of the right pane of the Activity Window, but no contents displayed in either pane, just a blank grey background.

Ths is very similar to the flakyness I had last week when I ran 6.1. I realize these symptoms are very vague and may not help you much, but I think there is some bug in 6.1 that is causing the flakyness..I don't know why it only happens for me. 6.04 works great and has been for many months. With no changes to my Win2k server, 6.1 only works correctly for a few hours. I'll try to answer any specific questions you may have to help resolve this issue. Until these issues are resolved I'll stay on 6.04. Thanks, Steve

[adrien]

Hi Steve

Sounds like you are seeing an issue that has cropped up in our test lab lately with a handle leak or similar.

Were you running the Kaspersky AV plugin?

Adrien

[saubrey]

Yes, I was running KAV. I was also running Puresite

[saubrey]

I did notice that Wingate.exe G 6.1 was consuming 33 MB of VMSize (task manager), which is about 13 MB more than normal, but that extra 13 MB is not enough to cause my system to run out of memory. Maybe it ran out of NamedPipes or sockets or something...I didn't check for those.

Last week when my WG server was acting flaky, PSINFO (sysinternals.com), , issues from a client comptuer, failed to connect to the server. I'm guessing that PSINFO uses NamedPipes (just a wild guess) and there wasn't enough of them or something for PSINFO to connect.

[kgoodknecht]

I tested with reserving a port.

You need to use regedt32.exe instead of regedit.exe since you need to create a REG_MULTI_SZ value called ReservedPorts under

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\TCPIP\Parameters

need to reserve the range, so I added "1080-1080" and a blank line after.

Need to reboot after that.

Works a treat though. We put it into the installer, so next release will do this for us.

Adrien

Adrien,
This is not new to version 6, I had this same problem crop up three years ago, so I added the reg entry, only I reserverd 1080-1090 which gives me up to eleven proxy ports to use. Since the default SOCKS and Real Audio proxies default to these two ports it was a natural range.

[saubrey]

Is there any update on a fix for the resource leak that renders WG 6.1 unusable after a few hours?

-- Steve

[adrien]

Hi Steve

We did find an issue with SYN cookies (off by default in 6.1 and earlier for services bound to external interfaces, but now on by default) which could cause socket leaks if you were getting syn attacks or incomplete connection negotiations.

So can I suggest trying 6.1.1?

Cheers

Adrien

[saubrey]

I installed WG 6.1.1 four days ago and it is still running well. No resource leaks and no wierd behavior with WG as with 6.1. 6.1.1 seems to fix all the problems I was having with 6.1.

Thanks for finding and fixing this so quickly.

-- Steve

[adrien]

Hi Steve

Thanks for the feedback

Cheers

Adrien