Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04

Scharf, Michael (Nokia - DE)

Hi all,

 

As requested by the authors, this e-mail starts a working group acceptance call for draft-bagnulo-tcpm-generalized-ecn-04, which will run on the list until June 30.

 

The proposal is to add a new milestone to the  TCPM charter

 

  Submit document on adding Explicit Congestion Notification (ECN) to TCP Control Packets to the IESG for publication as Experimental RFC

 

and to use draft-bagnulo-tcpm-generalized-ecn as a starting point.

 

If you believe that TCPM should work on a document in this space, and in particular if you are willing to review it, please reply to this e-mails. Please also speak up if you have any concerns or objections.

 

Thanks

 

Michael


_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04

Padma Bhooma
Hi Marcelo and Bob,

I have reviewed the document draft-bagnulo-tcpm-generalized-ecn-04 and I am sending my comments below. Overall, I think there is value in working on these additions or enhancements to RFC 3168 in tcpm working group. I am willing to review and provide feedback as needed.

  • Section 3.2.1.3 - If the SYN-ACK does not support AccECN, TCP must conservatively reduce its IW and SHOULD reduce it to 1 SMSS. This clause is very conservative because AccECN support can be deployed on servers slowly. Even if both end hosts support AccECN, there are networks with performance proxies that will not allow the feedback to propagate correctly. On those networks, TCP connections with ECT on SYN will always start with 1MSS of IW. I think this is too conservative and might add multiple RTT to grow the window. I agree with the value in adding ECT marking to SYN and SYN-ACK in terms of the latency benefits that might bring to an end user. I think “pessimistic ECT and cache successes” strategy described in 4.2.1 sounds like a reasonable one.

  • Section 3.2.3 ECT marking on Pure Acks - It felt like the document suggested further discussion on setting ECT on pure Acks, please correct me if I am wrong here. Setting ECT on pure Acks can increase the load on an AQM that might already be under load. This is because the number of Acks can be large and the information conveyed to the peer is redundant because asks are cumulative. TCP acknowledgements also do not solicit a response from the peer most of the times. So the chances of not echoing CE on an Ack is high. It is also difficult to detect if an Ack is dropped by an incompatible middle box because of ECT marking on it since we do not retransmit Acks.. So the fallback mechanisms will be limited. In Section 4.4.1 (Cwnd response to CE-marking on Pure Acks), the document says that we can echo a CE received on a pure ack on a data segment that will be sent later. In general, I think any congestion notification is relevant only for a short period of time probably in the order of a typical RTT. Once that time window passes, it may not be correct to respond to a delayed congestion notification. So echoing CE from a pure Ack on a data segment sent much later may not be the correct thing to do.

  • Section 3.2.4 Window Probe - Adding ECT to window probes is really good because this traffic is not too large on any connection and it requires the peer to respond so that there is a way to echo the CE marking. One modification that might be worth discussing is to require congestion response to CE marking on a window probe only when the receive window is opened. I will try to explain why this is reasonable. If congestion at the bottleneck link is transient, then lowering the congestion window in response to CE on a window probe that does not open the receive window might not be necessary because there won’t be any data sent afterwards. This can actually happen multiple times on a connection because a connection can go on for a long time sending multiple window probes when the peer does not open the window. So reducing congestion window only when the receive window is opened as a response to CE seems to be accurate in terms of timing and reaction to congestion notification.

  • Section 3.2.7 Retransmissions - If a connection doesn’t set ECT on a SYN because it does not support AccECN feedback (due to fallback mechanisms or something else), then the first data packet will be the first packet on a connection with ECT marking on it. If it is dropped, sending retransmissions without ECT might make it go through if there is a middle box that incorrectly drops packets with ECT marking. I think, this detection and fallback mechanism must be somehow included in the proposal to mark retransmissions with ECT.

Another point to note is that a connection should be already in recovery and must have adjusted it’s congestion window before sending a retransmission because it detected loss. If it further detects that there was CE marking on a retransmitted packet then it is not clear if it needs to adjust the congestion window again. Adding some clarification here will be good.

  • Tail loss probe - I think it will also be useful to include some text about CE marking on probe packets that a connection might send in response to tail loss (draft-dukkipati-tcpm-tcp-loss-probe-01.txt).

  • Implementation or deployment related comments - The document refers to other drafts or experimental RFCs as possible actions that can be taken in several scenarios. For example AccECN, RFC 5690, RFC 7567 etc,. Even though implementing and deploying these specifications is possible, they may not be implemented unless these specifications are tightly bound together. A casual reader might understand that implementing multiple of these RFCS will make the system work well. But for someone to implement and deploy these specifications, it is not clear if some or all of these implementations will need to be supported. It will be good to make a recommendation for the minimum set of specifications that need to be implemented — may be after some more discussion.

  • Fallback mechanisms - The document also discusses several fallback mechanisms to handle incompatibilities in the network. If we want implementations to consider these cases, it will be useful to formalize these detection and fallback mechanisms. I found that some of these mechanisms are being discussed in section 4 which was tagged as informative.

Overall, the document is written well and discusses multiple aspects of the proposed changes both on the end hosts and in the network. It is useful to have discussions on the proposed changes and to drive measurements that are needed to collect data to support or validate some of the assumptions made.

Thanks,
Padma




  1. Working group acceptance call
     draft-bagnulo-tcpm-generalized-ecn-04
     (Scharf, Michael (Nokia - DE/Stuttgart))


----------------------------------------------------------------------

Message: 1
Date: Fri, 16 Jun 2017 06:06:43 +0000
From: "Scharf, Michael (Nokia - DE/Stuttgart)"
<[hidden email]>
To: "[hidden email]" <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: [tcpm] Working group acceptance call
draft-bagnulo-tcpm-generalized-ecn-04
Message-ID:
<[hidden email]>

Content-Type: text/plain; charset="us-ascii"

Hi all,

As requested by the authors, this e-mail starts a working group acceptance call for draft-bagnulo-tcpm-generalized-ecn-04, which will run on the list until June 30.

The proposal is to add a new milestone to the  TCPM charter

 Submit document on adding Explicit Congestion Notification (ECN) to TCP Control Packets to the IESG for publication as Experimental RFC

and to use draft-bagnulo-tcpm-generalized-ecn as a starting point.

If you believe that TCPM should work on a document in this space, and in particular if you are willing to review it, please reply to this e-mails. Please also speak up if you have any concerns or objections.

Thanks

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailarchive.ietf.org/arch/browse/tcpm/attachments/20170616/151f93e9/attachment.html>

------------------------------

Subject: Digest Footer

_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm


------------------------------

End of tcpm Digest, Vol 158, Issue 7
************************************


_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04

Bob Briscoe-4
Padma,

On 17/06/17 17:27, Padma Bhooma wrote:
Hi Marcelo and Bob,

I have reviewed the document draft-bagnulo-tcpm-generalized-ecn-04 and I am sending my comments below. Overall, I think there is value in working on these additions or enhancements to RFC 3168 in tcpm working group. I am willing to review and provide feedback as needed.

  • Section 3.2.1.3 - If the SYN-ACK does not support AccECN, TCP must conservatively reduce its IW and SHOULD reduce it to 1 SMSS. This clause is very conservative because AccECN support can be deployed on servers slowly. Even if both end hosts support AccECN, there are networks with performance proxies that will not allow the feedback to propagate correctly. On those networks, TCP connections with ECT on SYN will always start with 1MSS of IW. I think this is too conservative and might add multiple RTT to grow the window. I agree with the value in adding ECT marking to SYN and SYN-ACK in terms of the latency benefits that might bring to an end user.
[BB] We have structured the document to just recommend the preferred approach in the normative spec (section 3), then discuss the rationale and possible alternatives in section 4. The sections just before 3.2.1.3 recommend:
  - 3.2.1.1: the client uses ECT optimistically
  - 3.2.1.2: but the client caches servers that don't recognize ECT.
This is the "optimistic ECT and cache failures" strategy discussed in 4.2.1 (tagged as 'S2B').

The requirement to reduce IW to 1SMSS only applies if the client is adopting this strategy and discovers the server does not support AccECN. Perhaps we didn't make that clear?

We have given 3 reasons for why this conservatism will rarely harm performance:

1) With this strategy, IW=1SMSS will only be needed the first time a client accesses a non-AccECN proxy or server.

2) During transition, this conservative IW=1SMSS will only ever be necessary for the client to server half-connection, not server to client.
According to Yuchung Cheng, client Web requests are rarely greater than 1 segment (no data openly available). Do you have any data to corroborate (or contradict) this? I think in non-Web apps, the initial client request would often fit within 1 packet too (e.g. email, ftp, ssh, scp, etc).

3) Fall-back to 1SMSS is written as SHOULD rather than MUST. And refers to text later that says under what conditions you can use a higher IW.

  • I think “pessimistic ECT and cache successes” strategy described in 4.2.1 sounds like a reasonable one.
[BB] What's your reasoning for preferring this one? We recommended "optimistic ECT and cache failures".


  • Section 3.2.3 ECT marking on Pure Acks - It felt like the document suggested further discussion on setting ECT on pure Acks, please correct me if I am wrong here. Setting ECT on pure Acks can increase the load on an AQM that might already be under load. This is because the number of Acks can be large and the information conveyed to the peer is redundant because asks are cumulative. TCP acknowledgements also do not solicit a response from the peer most of the times. So the chances of not echoing CE on an Ack is high.
[BB] Let's take these points one at a time:
PB: Setting ECT on pure ACKs could increase the load on an AQM
BB: An AQM processes packets of all sizes anyway. If pure ACKs are not ECT, it still has to decide whether to drop each pure ACK. Marking instead of dropping doesn't make any more work for the AQM.

PB: The information conveyed to the peer is redundant because ACKs are cumulative.
BB: Just because the ACKnumber info is superseded by the next ACK does not mean the ECN marking is superseded.

PB: TCP acknowledgements do not solicit a response from the peer most of the times. So the chances of not echoing CE on an Ack is high.
BB: Just because pure ACKs do not solicit a specific response (no ACK of an ACK) does not mean the data sender is not continuing to send data packets, on which ECE can be set (if RFC3168 feedback) or on which the CE counter can be increased (AccECN feedback).

RFC3168 actually says this:
   (One simple possibility would be for the sender to reduce its
   congestion window when it receives a pure ACK packet with the CE
   codepoint set).
Consider this pure ACK example with data solely in the S->C direction, where S is the server.
  * Say 3 in 1000 pure ACKs in the C->S direction are CE-marked, meaning the C->S marking probability is 0.3%
  * This might be because 0.3% of data packets in other flows (e.g. a YouTube upload) are being CE-marked as well.
  * So S needs to tell C that it has seen 3 CE marked packets (which it will do on the data packets it is sending, whether with RFC3168 feedback or AccECN feedback).
  * C is not (currently) sending any data to S, but it will reduce its cwnd for if it does start sending data.

  • It is also difficult to detect if an Ack is dropped by an incompatible middle box because of ECT marking on it since we do not retransmit Acks.. So the fallback mechanisms will be limited.
[BB] You're correct in all respects here. Nonetheless, I have three responses:
  * We have been doing millions of measurements over fixed and mobile networks, using a machine we control on both sides and checking a sent ECT packet is also received. So far, we have not seen any difference in the treatment of any control packet if set to ECT vs not-ECT.
  * Section 6.1.4 of RFC3168 hinted that in future pure ACKs might be set to ECT. So if a security middlebox is trying to enforce RFC3168, it still might allow ECT pure ACKs to pass.
  * Nonetheless, if there were a middlebox that dropped ECT pure ACKs, you are right that it could black-hole all ECT pure ACKs and stall a connection. And there would be no easy way to detect the specific cause of the problem and stop setting ECT on pure ACKs.
  * We should at least say that if fall-back has to be used because ECT SYNs or SYN-ACKs are not getting through, then ECT ought not to be used on pure ACKs either.

This is a good reason for why this protocol is experimental. However, given we have not so far found this problem in practice, I think it's OK to go ahead. But I agree that we should note that this is an open issue.

  • In Section 4.4.1 (Cwnd response to CE-marking on Pure Acks), the document says that we can echo a CE received on a pure ack on a data segment that will be sent later. In general, I think any congestion notification is relevant only for a short period of time probably in the order of a typical RTT. Once that time window passes, it may not be correct to respond to a delayed congestion notification. So echoing CE from a pure Ack on a data segment sent much later may not be the correct thing to do.
[BB] This is true.
However, this point applies generally to idle periods, not just when pure ACKs are marked. TCP can always enter an idle period just after a loss or CE mark is received, then if later the sender has something to send, cwnd will be lower. Cwnd is already reduced exponentially during an idle period.

This is all part of the fairly arbitrary rules on how to evolve cwnd during an idle period. Nonetheless, while idling, I think it is sensible to reduce cwnd more, if more of any pure ACKs sent during the idle period are picking up congestion markings. I don't think we should recommend the converse: i.e. if the congestion was a long time ago, we should not recommend that cwnd can be increased again.

However, this is an area where I think implementers should be allowed to experiment, and I think the specs allow enough flexibility to do so.



  • Section 3.2.4 Window Probe - Adding ECT to window probes is really good because this traffic is not too large on any connection and it requires the peer to respond so that there is a way to echo the CE marking. One modification that might be worth discussing is to require congestion response to CE marking on a window probe only when the receive window is opened. I will try to explain why this is reasonable. If congestion at the bottleneck link is transient, then lowering the congestion window in response to CE on a window probe that does not open the receive window might not be necessary because there won’t be any data sent afterwards. This can actually happen multiple times on a connection because a connection can go on for a long time sending multiple window probes when the peer does not open the window. So reducing congestion window only when the receive window is opened as a response to CE seems to be accurate in terms of timing and reaction to congestion notification.
[BB] On the other hand, if the window was constrained solely because the receive window was low and there had been a CE mark on a window probe many round trips ago, but not on more recent window probes, it would not be correct to reduce cwnd such a long time after the congestion had occurred.

I think it would be better to reduce cwnd immediately there is a CE on a window probe, increase cwnd as normal for every non-CE-marked window probe (even if rwnd is still the limiting factor). But also rely on cwnd validation so that cwnd cannot grow excessively if it is not being used.


  • Section 3.2.7 Retransmissions - If a connection doesn’t set ECT on a SYN because it does not support AccECN feedback (due to fallback mechanisms or something else), then the first data packet will be the first packet on a connection with ECT marking on it. If it is dropped, sending retransmissions without ECT might make it go through if there is a middle box that incorrectly drops packets with ECT marking. I think, this detection and fallback mechanism must be somehow included in the proposal to mark retransmissions with ECT.
[BB] Good point. I had figured that fall-back only needed to be considered for the SYN or SYN-ACK. But you're right that, if the SYN is not-ECT, fall-back might be needed later (altho, as already mentioned, we have not seen a need in measurements so far). This will generally only apply to the C->S half-connection.

This is a general point about fall-back that happens to be noticed when both data and retransmissions fail to get through. So I think we ought to add a general point about fall-back as a new subsection of section 3, rather than writing this in the retransmissions subjection. I propose this text:
3.2.8 General Fall-Back

Consider a host that has sent the first packet on a half-connection (SYN or SYN/ACK) without ECT set and it has been acknowledged. If this host's retransmission timer expires more than once for all subsequent packet(s) it sends with ECT set, it SHOULD retransmit the packet(s) with not-ECT set. If these not-ECT packet(s) are acknowledged without any further losses, it is RECOMMENDED that the host disables setting ECT on all subsequent packets of the half-connection. It could also cache this failure.




Another point to note is that a connection should be already in recovery and must have adjusted it’s congestion window before sending a retransmission because it detected loss. If it further detects that there was CE marking on a retransmitted packet then it is not clear if it needs to adjust the congestion window again. Adding some clarification here will be good.
[BB] Yes. We already say:
   If the TCP sender receives feedback that a retransmitted packet was
   CE-marked, it will react as it would to any feedback of CE-marking on
   a data packet.
Do you have a suggestion for clearer text? Perhaps:
   If the TCP sender receives feedback that a retransmitted packet was
   CE-marked, it will react as it would to any feedback of CE-marking on
   a data packet, in addition to its original reaction to the loss that 
   caused the retransmission.

RFC5681 already says:
   Loss in two successive windows of data, or the loss of a
   retransmission, should be taken as two indications of congestion and,
   therefore, cwnd (and ssthresh) MUST be lowered twice in this case.
It's hard to be more specific because we didn't want to contradict other congestion control schemes (e.g. some respond to at least one mark per RTT, others respond to the extent of CE marking).



  • Tail loss probe - I think it will also be useful to include some text about CE marking on probe packets that a connection might send in response to tail loss (draft-dukkipati-tcpm-tcp-loss-probe-01.txt).
[BB] Well, I would like to resist mission creep. The scope of our draft is intended to be about setting ECT on packets where it was previously prohibited.

We have included interactions with SCTP, TFO and IW10 in "5.  Interaction with popular variants or derivatives of TCP" because they have been published and can no longer be easily changed.

For TLP, it's not even adopted in TCP yet (although I'm sure it is widely used). So I think it's up to the authors of this draft to say whether ECT should be set on a TLP and what the response to a CE-marked TLP should be.


  • Implementation or deployment related comments - The document refers to other drafts or experimental RFCs as possible actions that can be taken in several scenarios. For example AccECN, RFC 5690, RFC 7567 etc,. Even though implementing and deploying these specifications is possible, they may not be implemented unless these specifications are tightly bound together. A casual reader might understand that implementing multiple of these RFCS will make the system work well. But for someone to implement and deploy these specifications, it is not clear if some or all of these implementations will need to be supported. It will be good to make a recommendation for the minimum set of specifications that need to be implemented — may be after some more discussion.
[BB] If my co-author agrees, we can do this. I would recommend the following complements (both normatively referenced):
* AccECN
* RFC5691 (robustness to blind in-window attacks).

As a reciprocal recommendation, currently, the Intro to the AccECN draft says:
   It is likely (but not required) that the AccECN protocol will be
   implemented along with the following experimental additions to the
   TCP-ECN protocol: ECN-capable TCP control packets and retransmissions
   [I-D.bagnulo-tcpm-generalized-ecn], which includes the ECN-capable
   SYN-ACK experiment [RFC5562]; and testing receiver non-compliance
   [I-D.moncaster-tcpm-rcv-cheat].


The following are good, but I don't think they qualify as complementary to ECN++:
* RFC5690 (ACKcc). It is merely optional and "not incompatible".
* Similarly for TFO and IW10. Optional but not incompatible.
* RFC7567 is about AQM in the network, so I don't it would help an implementer to say it is complementary, as host implementers cannot control where their implementation is used.
* Originally we were going to recommend RFC5562 (ECN+ i.e. ECT on SYN-ACK packets), but once we realized it was over-conservative, we wrote a less conservative alternative directly into this ECN++ draft.


More generally, I know some people are starting to get really confused over how to put together all the piecemeal modifications to ECN. I think that would require an informational "ECN roadmap" draft. No promises, but I guess I am someone who would be well-placed to know what to write in such a draft.



  • Fallback mechanisms - The document also discusses several fallback mechanisms to handle incompatibilities in the network. If we want implementations to consider these cases, it will be useful to formalize these detection and fallback mechanisms. I found that some of these mechanisms are being discussed in section 4 which was tagged as informative.

[BB] I have just checked, and I cannot find any case where there is fall-back text in section 4 that ought to be normative. If you found any, pls say where. We tried to ensure that the recommended approach was always in section 3, even if section 4 discussed (then rejected) alternatives.

Overall, the document is written well and discusses multiple aspects of the proposed changes both on the end hosts and in the network. It is useful to have discussions on the proposed changes and to drive measurements that are needed to collect data to support or validate some of the assumptions made.

[BB] Thanks.

In summary, as this conversation progresses, we might agree further changes. However, so far I have queued up the following list of changes to the text to address your points:
* Intro: Recommend complementary RFCs or drafts
* 3.2.1.3: Clarify that using IW=1SMSS only applies when using the strategies recommended in the previous two sections.
* When fall-back due to ECT on SYN (3.2.1.4) or SYN-ACK (3.2.2.3), consider disabling ECT on other control packets (esp. pure ACKs).
* Add detection of black-holing of pure ACKs as an open issue (hence why we have to take the approach in the previous bullet)
* Add extra section on general fall-back if no ECT packet has ever been ack'd
* And we will certainly acknowledge your contribution, and I hope you will continue to track changes to this draft carefully and let us know if you have implementation experience - thank you.


Bob


Thanks,
Padma




  1. Working group acceptance call
     draft-bagnulo-tcpm-generalized-ecn-04
     (Scharf, Michael (Nokia - DE/Stuttgart))


----------------------------------------------------------------------

Message: 1
Date: Fri, 16 Jun 2017 06:06:43 +0000
From: "Scharf, Michael (Nokia - DE/Stuttgart)"
<[hidden email]>
To: "[hidden email]" <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: [tcpm] Working group acceptance call
draft-bagnulo-tcpm-generalized-ecn-04
Message-ID:
<[hidden email]>

Content-Type: text/plain; charset="us-ascii"

Hi all,

As requested by the authors, this e-mail starts a working group acceptance call for draft-bagnulo-tcpm-generalized-ecn-04, which will run on the list until June 30.

The proposal is to add a new milestone to the  TCPM charter

 Submit document on adding Explicit Congestion Notification (ECN) to TCP Control Packets to the IESG for publication as Experimental RFC

and to use draft-bagnulo-tcpm-generalized-ecn as a starting point.

If you believe that TCPM should work on a document in this space, and in particular if you are willing to review it, please reply to this e-mails. Please also speak up if you have any concerns or objections.

Thanks

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailarchive.ietf.org/arch/browse/tcpm/attachments/20170616/151f93e9/attachment.html>

------------------------------

Subject: Digest Footer

_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm


------------------------------

End of tcpm Digest, Vol 158, Issue 7
************************************



_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04

Bob Briscoe-4
In reply to this post by Padma Bhooma
Padma, [re-sending - somehow my email client missed you off the distr.]

On 17/06/17 17:27, Padma Bhooma wrote:
Hi Marcelo and Bob,

I have reviewed the document draft-bagnulo-tcpm-generalized-ecn-04 and I am sending my comments below. Overall, I think there is value in working on these additions or enhancements to RFC 3168 in tcpm working group. I am willing to review and provide feedback as needed.

  • Section 3.2.1.3 - If the SYN-ACK does not support AccECN, TCP must conservatively reduce its IW and SHOULD reduce it to 1 SMSS. This clause is very conservative because AccECN support can be deployed on servers slowly. Even if both end hosts support AccECN, there are networks with performance proxies that will not allow the feedback to propagate correctly. On those networks, TCP connections with ECT on SYN will always start with 1MSS of IW. I think this is too conservative and might add multiple RTT to grow the window. I agree with the value in adding ECT marking to SYN and SYN-ACK in terms of the latency benefits that might bring to an end user.
[BB] We have structured the document to just recommend the preferred approach in the normative spec (section 3), then discuss the rationale and possible alternatives in section 4. The sections just before 3.2.1.3 recommend:
  - 3.2.1.1: the client uses ECT optimistically
  - 3.2.1.2: but the client caches servers that don't recognize ECT.
This is the "optimistic ECT and cache failures" strategy discussed in 4.2.1 (tagged as 'S2B').

The requirement to reduce IW to 1SMSS only applies if the client is adopting this strategy and discovers the server does not support AccECN. Perhaps we didn't make that clear?

We have given 3 reasons for why this conservatism will rarely harm performance:

1) With this strategy, IW=1SMSS will only be needed the first time a client accesses a non-AccECN proxy or server.

2) During transition, this conservative IW=1SMSS will only ever be necessary for the client to server half-connection, not server to client.
According to Yuchung Cheng, client Web requests are rarely greater than 1 segment (no data openly available). Do you have any data to corroborate (or contradict) this? I think in non-Web apps, the initial client request would often fit within 1 packet too (e.g. email, ftp, ssh, scp, etc).

3) Fall-back to 1SMSS is written as SHOULD rather than MUST. And refers to text later that says under what conditions you can use a higher IW.

  • I think “pessimistic ECT and cache successes” strategy described in 4.2.1 sounds like a reasonable one.
[BB] What's your reasoning for preferring this one? We recommended "optimistic ECT and cache failures".


  • Section 3.2.3 ECT marking on Pure Acks - It felt like the document suggested further discussion on setting ECT on pure Acks, please correct me if I am wrong here. Setting ECT on pure Acks can increase the load on an AQM that might already be under load. This is because the number of Acks can be large and the information conveyed to the peer is redundant because asks are cumulative. TCP acknowledgements also do not solicit a response from the peer most of the times. So the chances of not echoing CE on an Ack is high.
[BB] Let's take these points one at a time:
PB: Setting ECT on pure ACKs could increase the load on an AQM
BB: An AQM processes packets of all sizes anyway. If pure ACKs are not ECT, it still has to decide whether to drop each pure ACK. Marking instead of dropping doesn't make any more work for the AQM.

PB: The information conveyed to the peer is redundant because ACKs are cumulative.
BB: Just because the ACKnumber info is superseded by the next ACK does not mean the ECN marking is superseded.

PB: TCP acknowledgements do not solicit a response from the peer most of the times. So the chances of not echoing CE on an Ack is high.
BB: Just because pure ACKs do not solicit a specific response (no ACK of an ACK) does not mean the data sender is not continuing to send data packets, on which ECE can be set (if RFC3168 feedback) or on which the CE counter can be increased (AccECN feedback).

RFC3168 actually says this:
   (One simple possibility would be for the sender to reduce its
   congestion window when it receives a pure ACK packet with the CE
   codepoint set).
Consider this pure ACK example with data solely in the S->C direction, where S is the server.
  * Say 3 in 1000 pure ACKs in the C->S direction are CE-marked, meaning the C->S marking probability is 0.3%
  * This might be because 0.3% of data packets in other flows (e.g. a YouTube upload) are being CE-marked as well.
  * So S needs to tell C that it has seen 3 CE marked packets (which it will do on the data packets it is sending, whether with RFC3168 feedback or AccECN feedback).
  * C is not (currently) sending any data to S, but it will reduce its cwnd for if it does start sending data.

  • It is also difficult to detect if an Ack is dropped by an incompatible middle box because of ECT marking on it since we do not retransmit Acks.. So the fallback mechanisms will be limited.
[BB] You're correct in all respects here. Nonetheless, I have three responses:
  * We have been doing millions of measurements over fixed and mobile networks, using a machine we control on both sides and checking a sent ECT packet is also received. So far, we have not seen any difference in the treatment of any control packet if set to ECT vs not-ECT.
  * Section 6.1.4 of RFC3168 hinted that in future pure ACKs might be set to ECT. So if a security middlebox is trying to enforce RFC3168, it still might allow ECT pure ACKs to pass.
  * Nonetheless, if there were a middlebox that dropped ECT pure ACKs, you are right that it could black-hole all ECT pure ACKs and stall a connection. And there would be no easy way to detect the specific cause of the problem and stop setting ECT on pure ACKs.
  * We should at least say that if fall-back has to be used because ECT SYNs or SYN-ACKs are not getting through, then ECT ought not to be used on pure ACKs either.

This is a good reason for why this protocol is experimental. However, given we have not so far found this problem in practice, I think it's OK to go ahead. But I agree that we should note that this is an open issue.

  • In Section 4.4.1 (Cwnd response to CE-marking on Pure Acks), the document says that we can echo a CE received on a pure ack on a data segment that will be sent later. In general, I think any congestion notification is relevant only for a short period of time probably in the order of a typical RTT. Once that time window passes, it may not be correct to respond to a delayed congestion notification. So echoing CE from a pure Ack on a data segment sent much later may not be the correct thing to do.
[BB] This is true.
However, this point applies generally to idle periods, not just when pure ACKs are marked. TCP can always enter an idle period just after a loss or CE mark is received, then if later the sender has something to send, cwnd will be lower. Cwnd is already reduced exponentially during an idle period.

This is all part of the fairly arbitrary rules on how to evolve cwnd during an idle period. Nonetheless, while idling, I think it is sensible to reduce cwnd more, if more of any pure ACKs sent during the idle period are picking up congestion markings. I don't think we should recommend the converse: i.e. if the congestion was a long time ago, we should not recommend that cwnd can be increased again.

However, this is an area where I think implementers should be allowed to experiment, and I think the specs allow enough flexibility to do so.



  • Section 3.2.4 Window Probe - Adding ECT to window probes is really good because this traffic is not too large on any connection and it requires the peer to respond so that there is a way to echo the CE marking. One modification that might be worth discussing is to require congestion response to CE marking on a window probe only when the receive window is opened. I will try to explain why this is reasonable. If congestion at the bottleneck link is transient, then lowering the congestion window in response to CE on a window probe that does not open the receive window might not be necessary because there won’t be any data sent afterwards. This can actually happen multiple times on a connection because a connection can go on for a long time sending multiple window probes when the peer does not open the window. So reducing congestion window only when the receive window is opened as a response to CE seems to be accurate in terms of timing and reaction to congestion notification.
[BB] On the other hand, if the window was constrained solely because the receive window was low and there had been a CE mark on a window probe many round trips ago, but not on more recent window probes, it would not be correct to reduce cwnd such a long time after the congestion had occurred.

I think it would be better to reduce cwnd immediately there is a CE on a window probe, increase cwnd as normal for every non-CE-marked window probe (even if rwnd is still the limiting factor). But also rely on cwnd validation so that cwnd cannot grow excessively if it is not being used.


  • Section 3.2.7 Retransmissions - If a connection doesn’t set ECT on a SYN because it does not support AccECN feedback (due to fallback mechanisms or something else), then the first data packet will be the first packet on a connection with ECT marking on it. If it is dropped, sending retransmissions without ECT might make it go through if there is a middle box that incorrectly drops packets with ECT marking. I think, this detection and fallback mechanism must be somehow included in the proposal to mark retransmissions with ECT.
[BB] Good point. I had figured that fall-back only needed to be considered for the SYN or SYN-ACK. But you're right that, if the SYN is not-ECT, fall-back might be needed later (altho, as already mentioned, we have not seen a need in measurements so far). This will generally only apply to the C->S half-connection.

This is a general point about fall-back that happens to be noticed when both data and retransmissions fail to get through. So I think we ought to add a general point about fall-back as a new subsection of section 3, rather than writing this in the retransmissions subjection. I propose this text:
3.2.8 General Fall-Back

Consider a host that has sent the first packet on a half-connection (SYN or SYN/ACK) without ECT set and it has been acknowledged. If this host's retransmission timer expires more than once for all subsequent packet(s) it sends with ECT set, it SHOULD retransmit the packet(s) with not-ECT set. If these not-ECT packet(s) are acknowledged without any further losses, it is RECOMMENDED that the host disables setting ECT on all subsequent packets of the half-connection. It could also cache this failure.




Another point to note is that a connection should be already in recovery and must have adjusted it’s congestion window before sending a retransmission because it detected loss. If it further detects that there was CE marking on a retransmitted packet then it is not clear if it needs to adjust the congestion window again. Adding some clarification here will be good.
[BB] Yes. We already say:
   If the TCP sender receives feedback that a retransmitted packet was
   CE-marked, it will react as it would to any feedback of CE-marking on
   a data packet.
Do you have a suggestion for clearer text? Perhaps:
   If the TCP sender receives feedback that a retransmitted packet was
   CE-marked, it will react as it would to any feedback of CE-marking on
   a data packet, in addition to its original reaction to the loss that 
   caused the retransmission.

RFC5681 already says:
   Loss in two successive windows of data, or the loss of a
   retransmission, should be taken as two indications of congestion and,
   therefore, cwnd (and ssthresh) MUST be lowered twice in this case.
It's hard to be more specific because we didn't want to contradict other congestion control schemes (e.g. some respond to at least one mark per RTT, others respond to the extent of CE marking).



  • Tail loss probe - I think it will also be useful to include some text about CE marking on probe packets that a connection might send in response to tail loss (draft-dukkipati-tcpm-tcp-loss-probe-01.txt).
[BB] Well, I would like to resist mission creep. The scope of our draft is intended to be about setting ECT on packets where it was previously prohibited.

We have included interactions with SCTP, TFO and IW10 in "5.  Interaction with popular variants or derivatives of TCP" because they have been published and can no longer be easily changed.

For TLP, it's not even adopted in TCP yet (although I'm sure it is widely used). So I think it's up to the authors of this draft to say whether ECT should be set on a TLP and what the response to a CE-marked TLP should be.


  • Implementation or deployment related comments - The document refers to other drafts or experimental RFCs as possible actions that can be taken in several scenarios. For example AccECN, RFC 5690, RFC 7567 etc,. Even though implementing and deploying these specifications is possible, they may not be implemented unless these specifications are tightly bound together. A casual reader might understand that implementing multiple of these RFCS will make the system work well. But for someone to implement and deploy these specifications, it is not clear if some or all of these implementations will need to be supported. It will be good to make a recommendation for the minimum set of specifications that need to be implemented — may be after some more discussion.
[BB] If my co-author agrees, we can do this. I would recommend the following complements (both normatively referenced):
* AccECN
* RFC5691 (robustness to blind in-window attacks).

As a reciprocal recommendation, currently, the Intro to the AccECN draft says:
   It is likely (but not required) that the AccECN protocol will be
   implemented along with the following experimental additions to the
   TCP-ECN protocol: ECN-capable TCP control packets and retransmissions
   [I-D.bagnulo-tcpm-generalized-ecn], which includes the ECN-capable
   SYN-ACK experiment [RFC5562]; and testing receiver non-compliance
   [I-D.moncaster-tcpm-rcv-cheat].


The following are good, but I don't think they qualify as complementary to ECN++:
* RFC5690 (ACKcc). It is merely optional and "not incompatible".
* Similarly for TFO and IW10. Optional but not incompatible.
* RFC7567 is about AQM in the network, so I don't it would help an implementer to say it is complementary, as host implementers cannot control where their implementation is used.
* Originally we were going to recommend RFC5562 (ECN+ i.e. ECT on SYN-ACK packets), but once we realized it was over-conservative, we wrote a less conservative alternative directly into this ECN++ draft.


More generally, I know some people are starting to get really confused over how to put together all the piecemeal modifications to ECN. I think that would require an informational "ECN roadmap" draft. No promises, but I guess I am someone who would be well-placed to know what to write in such a draft.



  • Fallback mechanisms - The document also discusses several fallback mechanisms to handle incompatibilities in the network. If we want implementations to consider these cases, it will be useful to formalize these detection and fallback mechanisms. I found that some of these mechanisms are being discussed in section 4 which was tagged as informative.

[BB] I have just checked, and I cannot find any case where there is fall-back text in section 4 that ought to be normative. If you found any, pls say where. We tried to ensure that the recommended approach was always in section 3, even if section 4 discussed (then rejected) alternatives.

Overall, the document is written well and discusses multiple aspects of the proposed changes both on the end hosts and in the network. It is useful to have discussions on the proposed changes and to drive measurements that are needed to collect data to support or validate some of the assumptions made.

[BB] Thanks.

In summary, as this conversation progresses, we might agree further changes. However, so far I have queued up the following list of changes to the text to address your points:
* Intro: Recommend complementary RFCs or drafts
* 3.2.1.3: Clarify that using IW=1SMSS only applies when using the strategies recommended in the previous two sections.
* When fall-back due to ECT on SYN (3.2.1.4) or SYN-ACK (3.2.2.3), consider disabling ECT on other control packets (esp. pure ACKs).
* Add detection of black-holing of pure ACKs as an open issue (hence why we have to take the approach in the previous bullet)
* Add extra section on general fall-back if no ECT packet has ever been ack'd
* And we will certainly acknowledge your contribution, and I hope you will continue to track changes to this draft carefully and let us know if you have implementation experience - thank you.


Bob


Thanks,
Padma




  1. Working group acceptance call
     draft-bagnulo-tcpm-generalized-ecn-04
     (Scharf, Michael (Nokia - DE/Stuttgart))


----------------------------------------------------------------------

Message: 1
Date: Fri, 16 Jun 2017 06:06:43 +0000
From: "Scharf, Michael (Nokia - DE/Stuttgart)"
<[hidden email]>
To: "[hidden email]" <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: [tcpm] Working group acceptance call
draft-bagnulo-tcpm-generalized-ecn-04
Message-ID:
<[hidden email]>

Content-Type: text/plain; charset="us-ascii"

Hi all,

As requested by the authors, this e-mail starts a working group acceptance call for draft-bagnulo-tcpm-generalized-ecn-04, which will run on the list until June 30.

The proposal is to add a new milestone to the  TCPM charter

 Submit document on adding Explicit Congestion Notification (ECN) to TCP Control Packets to the IESG for publication as Experimental RFC

and to use draft-bagnulo-tcpm-generalized-ecn as a starting point.

If you believe that TCPM should work on a document in this space, and in particular if you are willing to review it, please reply to this e-mails. Please also speak up if you have any concerns or objections.

Thanks

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailarchive.ietf.org/arch/browse/tcpm/attachments/20170616/151f93e9/attachment.html>

------------------------------

Subject: Digest Footer

_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm


------------------------------

End of tcpm Digest, Vol 158, Issue 7
************************************



_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

ECN++ (draft-bagnulo-tcpm-generalized-ecn-04) and SCTP

Bob Briscoe-4
In reply to this post by Scharf, Michael (Nokia - DE)
Michael, Randall, Xuesong,

Are you considering implementing ECN in SCTP? I noticed that your initial SCTP/ECN draft prohibits ECN on similar control packets to those prohibited in TCP. So I wrote this section about transferability to SCTP into the ECN++ draft, which is targeted at TCP but also has a section on variants.
(The adoption call for the draft is below - deadline: 30 Jun).
   
I know the details of handshake and other control packets are different, but I think the principles are transferable.
Is this a correct understanding of the position wrt SCTP?


Cheers


Bob


-------- Forwarded Message --------
Subject: [tcpm] Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04
Date: Fri, 16 Jun 2017 06:06:43 +0000
From: Scharf, Michael (Nokia - DE/Stuttgart) [hidden email]
To: [hidden email] [hidden email]
CC: [hidden email] [hidden email]


Hi all,

 

As requested by the authors, this e-mail starts a working group acceptance call for draft-bagnulo-tcpm-generalized-ecn-04, which will run on the list until June 30.

 

The proposal is to add a new milestone to the  TCPM charter

 

  Submit document on adding Explicit Congestion Notification (ECN) to TCP Control Packets to the IESG for publication as Experimental RFC

 

and to use draft-bagnulo-tcpm-generalized-ecn as a starting point.

 

If you believe that TCPM should work on a document in this space, and in particular if you are willing to review it, please reply to this e-mails. Please also speak up if you have any concerns or objections.

 

Thanks

 

Michael


-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04

David Schinazi
In reply to this post by Scharf, Michael (Nokia - DE)
Hi,

I believe there is value in this work and I think it should be adopted by the working group.

Thanks,
David Schinazi


On Jun 15, 2017, at 23:06, Scharf, Michael (Nokia - DE/Stuttgart) <[hidden email]> wrote:

Hi all,
 
As requested by the authors, this e-mail starts a working group acceptance call for draft-bagnulo-tcpm-generalized-ecn-04, which will run on the list until June 30.
 
The proposal is to add a new milestone to the  TCPM charter
 
  Submit document on adding Explicit Congestion Notification (ECN) to TCP Control Packets to the IESG for publication as Experimental RFC
 
and to use draft-bagnulo-tcpm-generalized-ecn as a starting point.
 
If you believe that TCPM should work on a document in this space, and in particular if you are willing to review it, please reply to this e-mails. Please also speak up if you have any concerns or objections.
 
Thanks
 
Michael
_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm


_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ECN++ (draft-bagnulo-tcpm-generalized-ecn-04) and SCTP

G Fairhurst
In reply to this post by Bob Briscoe-4
I do have this on my list to review. I did not expect this draft to mention SCTP. I will certainly comment, but do not expect to see new protocol work for SCTP being done in TCPM.

Gorry 

On 23 Jun 2017, at 18:50, Bob Briscoe <[hidden email]> wrote:

Michael, Randall, Xuesong,

Are you considering implementing ECN in SCTP? I noticed that your initial SCTP/ECN draft prohibits ECN on similar control packets to those prohibited in TCP. So I wrote this section about transferability to SCTP into the ECN++ draft, which is targeted at TCP but also has a section on variants.
(The adoption call for the draft is below - deadline: 30 Jun).
   
I know the details of handshake and other control packets are different, but I think the principles are transferable.
Is this a correct understanding of the position wrt SCTP?


Cheers


Bob


-------- Forwarded Message --------
Subject: [tcpm] Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04
Date: Fri, 16 Jun 2017 06:06:43 +0000
From: Scharf, Michael (Nokia - DE/Stuttgart) [hidden email]
To: [hidden email] [hidden email]
CC: [hidden email] [hidden email]


Hi all,

 

As requested by the authors, this e-mail starts a working group acceptance call for draft-bagnulo-tcpm-generalized-ecn-04, which will run on the list until June 30.

 

The proposal is to add a new milestone to the  TCPM charter

 

  Submit document on adding Explicit Congestion Notification (ECN) to TCP Control Packets to the IESG for publication as Experimental RFC

 

and to use draft-bagnulo-tcpm-generalized-ecn as a starting point.

 

If you believe that TCPM should work on a document in this space, and in particular if you are willing to review it, please reply to this e-mails. Please also speak up if you have any concerns or objections.

 

Thanks

 

Michael


-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/
_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm

_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ECN++ (draft-bagnulo-tcpm-generalized-ecn-04) and SCTP

Bob Briscoe-4
Gorry,

Thx for promising to review.

The draft does not affect SCTP. We have included a section at the end on the (non-)interaction of Generalized ECN with TCP variants (SCTP, TFO, IW10, SYN cookies, L4S).
https://tools.ietf.org/html/draft-bagnulo-tcpm-generalized-ecn-04#section-5

We note that, currently, the only draft on adding ECN to SCTP mimics RFC3168 by prohibiting ECN support on all control packets. We then merely note that:
   "When ECN is finally added to SCTP, experience
   from experiments on adding ECN support to all TCP packets ought to be
   directly transferable to SCTP."

I know TFO, IW10, etc in depth. However, I was just checking with the authors whether this statement hides complexity in SCTP that I'm not aware of (cos I don't know SCTP in depth).


Bob

On 24/06/17 10:17, Gorry (erg) wrote:
I do have this on my list to review. I did not expect this draft to mention SCTP. I will certainly comment, but do not expect to see new protocol work for SCTP being done in TCPM.

Gorry 

On 23 Jun 2017, at 18:50, Bob Briscoe <[hidden email]> wrote:

Michael, Randall, Xuesong,

Are you considering implementing ECN in SCTP? I noticed that your initial SCTP/ECN draft prohibits ECN on similar control packets to those prohibited in TCP. So I wrote this section about transferability to SCTP into the ECN++ draft, which is targeted at TCP but also has a section on variants.
(The adoption call for the draft is below - deadline: 30 Jun).
   
I know the details of handshake and other control packets are different, but I think the principles are transferable.
Is this a correct understanding of the position wrt SCTP?


Cheers
Gorry

Bob


-------- Forwarded Message --------
Subject: [tcpm] Working group acceptance call draft-bagnulo-tcpm-generalized-ecn-04
Date: Fri, 16 Jun 2017 06:06:43 +0000
From: Scharf, Michael (Nokia - DE/Stuttgart) [hidden email]
To: [hidden email] [hidden email]
CC: [hidden email] [hidden email]


Hi all,

 

As requested by the authors, this e-mail starts a working group acceptance call for draft-bagnulo-tcpm-generalized-ecn-04, which will run on the list until June 30.

 

The proposal is to add a new milestone to the  TCPM charter

 

  Submit document on adding Explicit Congestion Notification (ECN) to TCP Control Packets to the IESG for publication as Experimental RFC

 

and to use draft-bagnulo-tcpm-generalized-ecn as a starting point.

 

If you believe that TCPM should work on a document in this space, and in particular if you are willing to review it, please reply to this e-mails. Please also speak up if you have any concerns or objections.

 

Thanks

 

Michael


-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/
_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

_______________________________________________
tcpm mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/tcpm
Loading...