Socket Errors and Dropped Connections

[DBeard]

Just to reiterate the problem in a new thread...


I'm still experiencing periodically, the Wingate server fails the respond to request, giving browsers a Socket Error and booting all SSL, Citrix, and Socks connections. This problem occurs aprox 1-10 times an hour and last for about 30 seconds to 2 minutes.

Any word on the upcoming patch release date and if it will address this problem.

So far I've owned this software for about a month and have yet to be able to keep it in production. I'm really getting chewed over this. Please address this quickly. Thanks

[DBeard]

bump

[labull]

Don't know you've tried so I'll throw this in.

Have you deleted the Histroy.dbf and History.cdx files? These getting corrupted can be a source of WinGate hangs.


Larry

[DBeard]

Yes in fact, because sometimes it stops reporting on the history page and this is my solution.

[DBeard]

bump

Word on patch??????????????

[DBeard]

Look I can't wait any longer for you to address this problem.

[adrien]

Hi

Sorry to be away from this forum for a while, I hope you also got my email.

What particular socket error do you get?

Adrien

[DBeard]

You would receive the same socket error page if there where an interupted connection between the browser request for a page and Wingate not finding the page before time out.

The browser reports Socket Error in big letters and something along the lines of the host connection timed out.

The other symptom that I watch is Socks instant messenger services, such as MSN, Yahoo, or AIM, will drop their connections at the same moment that HTTP traffic is refused or lost.

I set me instant messengers to make a sound effect when they loose connection, which alerts me to have a look at the proxy server, which each time will be just spinning it's wheels. Not locked up, as I can launch Gatekeeper and have a look around the console, but it won't accept any incoming connections from browsers or anything else. Typically lasts for about 30 seconds. Just long enough to terminate Citrix and other SSL type connection modules.

[chespir]

are those messages something likt this?:

11/03/03 12:44:29 192.168.10.40 Guest 0000000159 Error: Caught socket exception in CWWWSession::HTTPProcessRequest() Socket Error 10049 {Thd 484} [socket #AF0] - terminating
11/03/03 12:44:29 192.168.10.40 Guest 0000000159 Traffic 222 441 0 0 0s
11/03/03 12:44:29 192.168.10.40 Guest 0000000159 Terminated exit code 1


my wingate looks like asking to the dns, then suddndly quits loading the page.

[DBeard]

Not nearly that detailed a description.

just Socket Error

As soon as it occurs again, I'll let you know exactly.

The good news is, since the update, so far only one drop all morning.

[DBeard]

Spoke to soon. Getting socket errors every minute now.

Here's what the browser will report.

Socket Error

Connection to Remote Host timed out

[chespir]

...but you must look for the info in the WWW log, nos what the explorer shows when missing the page. Go to whatever\wingate\logs\WWW\"today" and see what error does it print in the .txt.

[DBeard]

WWW logs
0000005970 Error: Caught socket exception in CWWWSession::HTTPProcessRequest() Connection to Remote Host timed out - terminating

Socks does not report an error, it just drops connections.

Keep in mind, I test the Internet Service connection at the very moment that these are happening, by attaching to the gateway without going through the proxy to ensure it is not our ISP.

[adrien]

How is your WWW proxy configured to make connections?

On the connections tab is it set to connect directly (top option)?

Because you have the Interfaces set, there could be a problem with using the multiple NICs that you have configured.

If you specify on the Interfaces tab in the WWW proxy "connections out will be made on any interface" rather than binding to a specific interface, then that should remove that as a possible source of problems

[DBeard]

Remember that I have an interface that is a second gateway for incoming HTTP traffic from the web.

When I set the settings as you describe won't Wingate get confused on which gateway it should use?

[chr]

This is perhaps a stupid sugesstion but check your Wingate for viruses. I manage a installation with 30-40 users and suddenly it went down, running fine for a couple of hours and then "chokes". Since i blocked all port to the internet viruses was not my first thought. After some time of testing i noticed that there was a worm and it was delivered from the inside. Even if you think that you know your network and your users you cant be sure.

[DBeard]

security analysis is clean, no viruses.

[adrien]

Hi

I think whenever you have 2 default gateways where each gateway has a NAT you are going to have problems. The OS will always choose the highest priority default gateway for outbound packets, this means that packets received by either gateway will generate a response packet from WinGate that will be forwarded to only one of the gateways.

This will break NAT connections, since each NAT must receive inbound and outbound packets from the connections that it maintains.

For instance, if you have a bunch of connections going through gateway 1, then you turn gateway 2 on, gateway 2 is going to suddenly receive all the outbound packets from the existing connections going through gateway 1. Since any NAT will only create a hash entry for TCP SYN packets, and since these connections will already have been set up, it would not surprise me if the second gateway took a dim view of these packets, and maybe even send FIN packets to trash the connections... or otherwise the gateway 1 would time out the connections, since it wouldn't be receiving any more packets to keep the connections alive.

I am trying to think of a way around this for you. Do the clients that connect in through the other NAT (192.168.0.x subnet) have known IP addresses? If so, you could simply set up specific routes to these IP addresses, and not assign a default gateway to your 192.168.0.X interface. Then when these IPs connect, you will not mess up your routing, and the packets will go back out through the correct gateway, and since you wouldn't have 2 default gateways, then it wouldn't mess up your existing connections either.

Adrien

[DBeard]

Do the clients that connect in through the other NAT (192.168.0.x subnet) have known IP addresses?
No, these are typical anonymous web site traffic from various locations.

For instance, if you have a bunch of connections going through gateway 1, then you turn gateway 2 on, gateway 2 is going to suddenly receive all the outbound packets from the existing connections going through gateway 1.

New to TCP2 is Metric. The ability to set gateway priority for multiple gateways on the same system. The lower the metric (default =1) the greater the priority the OS will put on the gateway. So in theory, if a connection is found on the first gateway (DNS), with the lowest metric, traffic would never hit the second gateway.

I need to reiterate here, that I have tested extensively, the OS system that the Proxy is on. The OS itself never has the problem that wingate has. IOW, when wingate chokes, the OS is still able to function around the proxy server.

To explain: I setup the browser on the Wingate server to function through the default gateway, without going to the proxy server. I then use Sniffer to trace the packets to see where they are going and why.

Never does an outbound packet hit the second 192 gateway. Not at all.

In conclusion, I can say with confidence, that this is not related to the OS in any way, if there is a problem with dealing with multiple interfaces, I believe the problem is wingates.

[DBeard]

The latest patch has worsened the problem.
I can not use wingate at all anymore.

Socket Errors, Dropped Connections or DNS lookup problems, however you refer to them, they occur EVERY MINUTE now.

I can't even begin to convey how frustrated I am with Wingate.

[chespir]

D. BEARD do you REALLY trust windows managing more than one gateway, and handling metrics? Thats crazy. You should keep one clear gateway if you expect windows to handle traffic. Did you try that?

All my problems till now where due to routing issues and/or wingate miss-configurations, already solved (til next :))) I have +-100 users going out through some services plus some coming in through vpn.

One little question to adrien, is it really woth the upgrade for stable systems?

thanks for last advices anyway adrien.

[DBeard]

I trust them entirely because I KNOW they work properly when properly configured.

The problem with routes in Windows OS will always be a directly relation to how well the routes have been set by the administrator. The OS is not at fault.

The problem here is not how windows handles routing, but how Wingate handles multiple interfaces.

[adrien]

New to TCP2 is Metric. The ability to set gateway priority for multiple gateways on the same system. The lower the metric (default =1) the greater the priority the OS will put on the gateway. So in theory, if a connection is found on the first gateway (DNS), with the lowest metric, traffic would never hit the second gateway.

I need to reiterate here, that I have tested extensively, the OS system that the Proxy is on. The OS itself never has the problem that wingate has. IOW, when wingate chokes, the OS is still able to function around the proxy server.

To explain: I setup the browser on the Wingate server to function through the default gateway, without going to the proxy server. I then use Sniffer to trace the packets to see where they are going and why.

Never does an outbound packet hit the second 192 gateway. Not at all.

In conclusion, I can say with confidence, that this is not related to the OS in any way, if there is a problem with dealing with multiple interfaces, I believe the problem is wingates.

OK, however WinGate without ENS is simply a sockets application, and relies entirely on winsock for connectivity. I believe later versions of IE don't always use winsock (since sometimes they don't load our client LSP). A better test may be an FTP client application - try connecting to a site, starting a large download, and enabling your second interface with its default gateway as well.

Is the metric the same for the second default route?

Windows uses normally the lowest metric.

If outbound packets are never going to your second gateway, why even use it? Wouldn't you be better off pointing these people to your main gateway? The one WinGate connects out from?

I think with Win2000 MS tried to be smart with multiple default routes, but unless you associate a route with a socket, you can't be certain what will be used. And I'm pretty sure windows doesn't do that.

If multiple default routes are so great, why do XP and 2k3 server moan so much if you get more than one?

I think MS learned their lesson with Win2k and decided multiple default routes is a very bad idea (I have to agree with them on that one).

[adrien]

PS, are you still using the interface tab in the WWW Proxy?

Adrien

[chespir]

drbeard try this little thing for me: set two gateways with different metrics (which i think you already have) and then unplug the destination of the one with highest priority (lowest metric), and see what happens. Do you REALLY trust windows handling your routing work?

[DBeard]

Chespir

Let me repeat myself and include XP into this discussion, even though it is not part of my problem.

On any MS OS 2000 and later, with TCP 2 installed, Metrics and routing tables with persistent routes configured YES it works perfectly everytime for me. (routing tables are VERY important)

If you are having some trouble, I suggest your configuration is incorrect or your route table is wrong.

I can try your test over and over and over with XP (as example). I have three NICs in the machine from which I am typing right now. Two are connected to the same network with different gateways (one private one public), the other is attached to a different subnet entirely. I have Metrics set for each and routing tables configured in detail. I can unplug any one of the NICs, whether it be the one with the highest or lowest metric and I will still be able to properly route to whatever (internet or local addresses) as long as the local address is on the network that at least one NIC is attached. Additionally I can unplug TWO if I care to further my point, and continue to funtion as I would expect. Tracing packets finds that they are indeed visiting the Metriced Gateways in order.

I further my point by adding I have multiple gateways configured on one of these three NICs with Metrics set, and again packets visit the lowest Metric first, failing to find the destination moving on to the next. It works EVERYTIME.

Yes I have experienced what youre aiming at, but only when my route table is misconfigured.

****

Now back to the problem. I have lost confidence that this issue will be resolved. We've now stopped focusing on your product and begun to point fingers and place blame elsewhere.

Obviously Wingate works in a default configuration of it's intended use or a very slight variation. But any deviation from that default and you're gonna have a problem.

There is obviously a solution to this issue, because my previous proxy funtions in this capacity (all be it, having a lack of integrated authentication, which is why I bought your product to begin with).

****

As I have said, there is something in the way that Wingate binds up interfaces, even when you can netstat and see that nothing is listening on whatever port.

ANY other product utilizing the same interface, although excluded in the Gatekeeper GUI, will report the address in use on the port on which Wingate runs a service. This includes, but isn't limited to IIS, U FTP, Bulletproof, Winbroker, and the list goes on.

There is an obvious problem in the way that Wingate excludes interfaces, NICs, or IP addresses that will prevent it from functioning properly in any cofiguration deviating from the default. Is this related to the engine freezing every 1 to 10 minutes, I don't know. But, just more information that seems suspicious.

****

To answer your question Adrien, YES. And the Binding tab. Not just on the WWW service but on all services.

In the end, I believe this is a deficiency in that Wingate does not utilize the metric of the OS, but rather forces packets to where it wants them to go. Its probably is fighting with the OS over the best route. JMO.

[chespir]

don´t get mad at me :)

i just ask because i´ve always had traumatic experiences with windows routing, never worked as i wanted it to. I just can´t make windows see a second gateway when the first one falls down. Anyway, probably i have not investigated it enough. I need a vpn solution in windows because of the external users, all af them have windows sistems, that´s why i got into vpn, wich worked right 99% of the time, except for some windows or windows based programs that collapsed dinamic routing through the vpn.

That´s why i was asking you that, cause it surprised me a lot :)
id you ever tried any routing software for windows? maybe it´s better for you to handle routing via software instead of via OS. I don´t know, just wondering.

again excuse my english.

[DBeard]

:P

I'm not mad, I'm just American. I get straight to the point with no pleasantries. I don't intend to have an overtone of anger. I'll admit frustration, but I'm simply trying to state my facts and have them understood, clearly.

On forums it's easy to mistake the tone of someones message, because you don't see their expression when making the comments. Don't hold it against me that I have no sense of humor :)

To answer you question, I've never handle routing in any other way than with the Windows OS.

[adrien]

I really suggest you don't set anything in the interfaces tab and see if it still does the same thing.

WinGate uses MSG_DONTROUTE on sends if you have it bound to a specific adapter.

Never seen this option have any effect, but it is a while since I looked, and since then things may have changed.

If you just use the top option on the Interfaces tab, then the default winsock behaviour should result, which if you are seeing the OS try multiple gateways, well it should do the same.

My final caution - beware that connections in may be treated differently to connections out by the OS. In your case with multiple interfaces receiving connections using different gateways, that may come into play vis a vis your test, which is outbound from the machine itself.

[tim]

:P
I'm not mad, I'm just American. ...
On forums it's easy to mistake the tone of someones message, because you don't see their expression when making the comments.

Maybe I should add the smilies back in :-)

Tim