Improvements suggestions

[RM_Goff]

Hi, Adrien and all Qbic stuff!

We use wingate since many years and fully happy with it. Now I have some improvement suggestion for WIngate. Hope it helps to make WG even better))

We use POP3 Collection Service. Our mailboxes are hosted at provider site (we do not want to host own mail server). Wingate collects new incoming messages using POP3 and relays them to our internal mail server. Everything works fine, but we miss two things badly.

1. Sometimes, 2-3 times per week POP3 collection hangs with one or more particular mailboxes. You just see freezed "POP3 Collection: Collecting mail from ..." line in System Activity window. At this time all other mailboxes work well, POP3 collection service connects and delivers to internal server, but this particular mailbox "hangs" - no new messages comes to internal server. There is only one way to repair it - you need to restart a whole wingate engine. Restarting only a POP3 Collection Service does not help - the whole wingate has to be restarted. That would be great to have a possibility to "drop" a particular POP3 collection job or to reset POP3 collection service without restarting wingate.

2. SMTP delivery service and Delivery Mail Settings. When you use POP3 Collection service, it relays collected mails to internal mailserver using SMTP delivery service. At the same time outgoing mails catched with SMTP server on internal NIC will be delivered to addressee using same service through external Interface. This are two different tasks (incoming and outcoming traffic), but you are not able to set two different sets of settings for them. It is not possible to set two independent SMTP delivery services and two independent Delivery settings in Mail Settings - one for Incoming mails (from external NIC to internal) and another for outgoing (from internal NIC to external). It would be much nicer to have a possibility to have several different settings for different purposes. For example, Mail Settings has to be moved to SMTP Delivery Service Properties and there must be possibility to install several independent SMTP delivery services with different settings for different purposes.

[adrien]

Hi

Thanks for your suggestions.

POP3 collection hanging.... I'd be keen to look into this further to see if we can eliminate the hang... what version of WinGate is this? Do you know if it recovers on restart because it skips the message it hung on last time? Do mails accumulate in the hosted mailbox and not get cleared out?

As for multiple SMTP delivery services. This is difficult, since there is only 1 SMTP queue, and delivery is from that.

However there are some ways to get different settings depending on the destination. For example if you define a server (in mail settings services tab) you can specify things like auth etc.

What sort of other settings would you like to be able to set based on destination? We could probably do something like push an event prior to delivery to allow policy to set configuration based on destination domain etc.

Adrien

[RM_Goff]

We use last version of WG and keep it up to date. It seems POP3 Collection Service hangs with particular mailboxex on a stage of negotiation with POP3 server, probably because of SSL issues. It happens randomly. We have somewhere 100 accounts on 2 different POP3 servers, collection task runs every 2 minute. There are no patterns to notice and it is impossible to predict which mailbox will hang and when it happens next time. May be it is not WG issue, may be it's providers. Mails are not lost - all mails will be downloaded and delivered to internal mail server after WG restart, "bad" message included. After successful downloading of messages they will be automatically deleted on external POP3 server.

I think, it is not because of the messages, it is because of connection/negotiation problems.

I changed log level to debug - let's wait if it helps to bring some clearness.

What about SMTP delivery - then it would be great to have more than one SMTP queue. There is a difference between incoming and outgoing mails. Now, for exаmple, it is not possible to use external (provider's) SMTP server to deliver outgoing mails and POP3 collection at the same time. It is tricky to assign particular internal NIC (if you have many) to deliver internal mails and particular external NIC to deliver outgoing (here works your advice to change metric in routing table but it is on OS level, not a WG). I may imagine more complex configurations (for example with several internal mail servers, different internal segments or even with mail servers in DMZ) that require more precise settings for each delivery direction.

[RM_Goff]

Hi, Adrien!

About POP3 Collection. Here is a part of debug log for POP3 Collection Service


2256401 06.05.2015 13:42:06.539 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: +OK POP server ready H mieue123 0M3xI2-1Z7u4Z3i0g-00qwSa
2256402 06.05.2015 13:42:06.539 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de C=>: CAPA
2256403 06.05.2015 13:42:06.570 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: +OK Capability list follows
2256404 06.05.2015 13:42:06.570 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: TOP
2256405 06.05.2015 13:42:06.570 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: UIDL
2256406 06.05.2015 13:42:06.570 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: USER
2256407 06.05.2015 13:42:06.570 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: SASL PLAIN
2256408 06.05.2015 13:42:06.570 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: IMPLEMENTATION trinity
2256409 06.05.2015 13:42:06.570 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: .
2256410 06.05.2015 13:42:06.570 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de C=>: AUTH PLAIN ********
2256411 06.05.2015 13:42:06.617 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: +OK mailbox "xxxxx@xxxxxxxx.de" has 5 messages (12675324 octets) H mieue123
2256412 06.05.2015 13:42:06.617 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de C=>: STAT
2256415 06.05.2015 13:42:06.648 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: +OK 5 12675324
2256416 06.05.2015 13:42:06.648 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de C=>: UIDL 1
2256428 06.05.2015 13:42:06.679 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: +OK 1 0ME3Yt-1XJ2cD301P-00HLCA
2256429 06.05.2015 13:42:06.679 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de C=>: UIDL 2
2256432 06.05.2015 13:42:06.710 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: +OK 2 0MFcST-1XHGtT3jVN-00EehM
2256433 06.05.2015 13:42:06.710 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de C=>: UIDL 3
2256440 06.05.2015 13:42:06.742 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: +OK 3 0MQsFq-1WtD1W48lC-00UH4i
2256441 06.05.2015 13:42:06.742 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de C=>: UIDL 4
2256456 06.05.2015 13:42:06.773 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: +OK 4 0MdjUe-1Y2D731lds-00PKCK
2256457 06.05.2015 13:42:06.773 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de C=>: UIDL 5
2256465 06.05.2015 13:42:06.804 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: +OK 5 0MK6AH-1YhLaG0NAz-001U4M
2256466 06.05.2015 13:42:06.804 127.0.0.2 Administrator POP3 Collection service 1684 158486 info 3 Request: POP3 Collection: no new messages for xxxxx@xxxxxxxx.de@pop.xxxxxxx.de
2256467 06.05.2015 13:42:06.804 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de C=>: QUIT
2256469 06.05.2015 13:42:06.835 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 0 xxxxx@xxxxxxxx.de@pop.xxxxxxx.de <=S: +OK POP server signing off
2256471 06.05.2015 13:42:06.835 127.0.0.2 Administrator POP3 Collection service 1684 158486 info 2 Traffic: 115 426 0 0 3s
2256472 06.05.2015 13:42:06.835 127.0.0.2 Administrator POP3 Collection service 1684 158486 debug 10 Session terminated: completed processing


It looks pretty OK, but after this point this mailbox "hangs" - POP3 Collection will not check this mailbox anymore until WG is restarted. And all this time until reboot a string "POP3 Collection: Collecting mail from xxxxx@xxxxxxxx.de@pop.xxxxxxx.de" is seen in System Activity window. All other mailboxes works fine at this time.

[adrien]

ok thanks for that. Very odd.

We do have code that prevents scheduling another check on a mailbox while a previous check is still running. I wonder if the session isn't coming down properly and freeing that up so the next one can proceed. I'll take a look in that area of the code.

Thanks

Adrien

[RM_Goff]

Thank you, Adrien!

Here is another sample, another mailbox that "hangs"

388543 12.05.2015 13:41:36.313 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: +OK POP server ready H mieue123 0MK6pZ-1YsxNv1pTx-0020zx
388545 12.05.2015 13:41:36.313 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de C=>: CAPA
388558 12.05.2015 13:41:36.360 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: +OK Capability list follows
388559 12.05.2015 13:41:36.360 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: TOP
388560 12.05.2015 13:41:36.360 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: UIDL
388561 12.05.2015 13:41:36.360 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: USER
388562 12.05.2015 13:41:36.360 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: SASL PLAIN
388563 12.05.2015 13:41:36.360 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: IMPLEMENTATION trinity
388564 12.05.2015 13:41:36.360 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: .
388565 12.05.2015 13:41:36.360 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de C=>: AUTH PLAIN ********
388582 12.05.2015 13:41:36.391 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: +OK mailbox "xxxx@xxxx.de" has 1 messages (27646 octets) H mieue123
388583 12.05.2015 13:41:36.391 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de C=>: STAT
388595 12.05.2015 13:41:36.422 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: +OK 1 27646
388596 12.05.2015 13:41:36.422 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de C=>: UIDL 1
388604 12.05.2015 13:41:36.453 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: +OK 1 0MYqrj-1YjRHq3V8V-00Vf36
388607 12.05.2015 13:41:36.453 127.0.0.2 Administrator POP3 Collection service 1772 32684 info 3 Request: POP3 Collection: no new messages for xxxx@xxxx.de@pop.xxxxxx.de
388608 12.05.2015 13:41:36.453 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de C=>: QUIT
388617 12.05.2015 13:41:36.485 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 0 xxxx@xxxx.de@pop.xxxxxx.de <=S: +OK POP server signing off
388618 12.05.2015 13:41:36.485 127.0.0.2 Administrator POP3 Collection service 1772 32684 info 2 Traffic: 79 289 0 0 1s
388619 12.05.2015 13:41:36.485 127.0.0.2 Administrator POP3 Collection service 1772 32684 debug 10 Session terminated: completed processing


And here one that works fine


1220457 12.05.2015 20:14:00.811 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: +OK POP server ready H mieue125 0M5aFU-1Z7jEu2oH2-00xywb
1220459 12.05.2015 20:14:00.811 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de C=>: CAPA
1220471 12.05.2015 20:14:00.843 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: +OK Capability list follows
1220473 12.05.2015 20:14:00.843 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: TOP
1220476 12.05.2015 20:14:00.843 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: UIDL
1220478 12.05.2015 20:14:00.843 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: USER
1220480 12.05.2015 20:14:00.843 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: SASL PLAIN
1220482 12.05.2015 20:14:00.843 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: IMPLEMENTATION trinity
1220484 12.05.2015 20:14:00.843 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: .
1220486 12.05.2015 20:14:00.843 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de C=>: AUTH PLAIN ********
1220499 12.05.2015 20:14:00.874 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: +OK mailbox "yyyyyy@yyyyyyy.de" has 5 messages (12675324 octets) H mieue125
1220500 12.05.2015 20:14:00.874 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de C=>: STAT
1220513 12.05.2015 20:14:00.905 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: +OK 5 12675324
1220515 12.05.2015 20:14:00.905 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de C=>: UIDL 1
1220552 12.05.2015 20:14:00.936 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: +OK 1 0ME3Yt-1XJ2cD301P-00HLCA
1220553 12.05.2015 20:14:00.936 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de C=>: UIDL 2
1220576 12.05.2015 20:14:00.967 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: +OK 2 0MFcST-1XHGtT3jVN-00EehM
1220577 12.05.2015 20:14:00.967 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de C=>: UIDL 3
1220593 12.05.2015 20:14:00.999 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: +OK 3 0MQsFq-1WtD1W48lC-00UH4i
1220595 12.05.2015 20:14:00.999 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de C=>: UIDL 4
1220610 12.05.2015 20:14:01.030 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: +OK 4 0MdjUe-1Y2D731lds-00PKCK
1220612 12.05.2015 20:14:01.030 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de C=>: UIDL 5
1220618 12.05.2015 20:14:01.061 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: +OK 5 0MK6AH-1YhLaG0NAz-001U4M
1220619 12.05.2015 20:14:01.061 127.0.0.2 Administrator POP3 Collection service 2104 101581 info 3 Request: POP3 Collection: no new messages for yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de
1220620 12.05.2015 20:14:01.061 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de C=>: QUIT
1220632 12.05.2015 20:14:01.092 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 0 yyyyyy@yyyyyyy.de@pop.yyyyyyyy.de <=S: +OK POP server signing off
1220633 12.05.2015 20:14:01.092 127.0.0.2 Administrator POP3 Collection service 2104 101581 info 2 Traffic: 115 426 0 0 1s
1220634 12.05.2015 20:14:01.092 127.0.0.2 Administrator POP3 Collection service 2104 101581 debug 10 Session terminated: completed processing


I see no difference :((