Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Brian E Carpenter-2
Hi,

Does the IESG plan to catch up on old reported errata that have never been processed?

There are three here for example: https://www.rfc-editor.org/errata/rfc6275, as much as 4 years old. There may be a lot more lurking.

Regards
   Brian Carpenter

On 08-May-21 04:38, IESG Secretary wrote:

> IESG Processing of RFC Errata for the IETF Stream
>
> Online: <https://www.ietf.org/about/groups/iesg/statements/processing-errata-ietf-stream/>
>
> This document describes how the IESG processes RFC Errata for the IETF Stream.
>
> These are strong guidelines, not immutable rules. The Area Directors (ADs) should use
> common sense and good judgment to decide what the right thing to do is. They apply to new
> errata and not to errata that have already been processed.
>
> Errata are meant to fix "bugs" in the specification and should not be used to change what the
> community meant when it approved the RFC.  Here are some things to consider when
> submitting an errata report:
>
> * Errata are items that were errors at the time the document was published -- things that
>   were missed during the last call, approval, and publication process.  If new information,
>   new capabilities, or new thinking has come up since publication, or if you disagree with
>   the content of the RFC, that is not material for an errata report.  Such items are better
>   brought to relevant working groups, technical area discussions, or the IESG.
> * Errata reports are usually for errors in the text version of a document.  It is possible to
>   report errors in other outputs (e.g., HTML or PDF) for RFCs published in the v3 format
>   (i.e., RFC 8650+).
> * Errata are classified as "technical" or "editorial".  Please mark the report appropriately.  
>   Technical errata are expected to be things that would likely cause significant
>   misunderstandings of the technical specification and might result in faulty
>   implementations if they are not corrected. Editorial errata are, as the name implies,
>   editorial - for example, typos, missing commas, etc. Errors in examples will generally be
>   editorial, though marking them as technical could sometimes be justified.
> * Please clearly explain the issue in the Comments section. This is especially important for
>   editorial issues, where the Original Text and Corrected Text may look almost identical.
>
> When a technical erratum is reported, a report is sent to the authors, chairs, and Area Directors
> (ADs) of the WG in which the document originated. If the WG has closed or the document was
> not associated with a WG, then the report will be sent to the ADs for the Area most closely
> associated with the subject matter.  
>
> When an editorial erratum is reported, the RFC Editor will do an initial review and handle errata
> that are clearly editorial in nature. If the erratum cannot be handled by the RFC Editor, the AD
> will be asked to review.
>
> While ADs are ultimately responsible for processing the reports, they may delegate the review
> or perform it personally.  The reviewer will classify the erratum as falling under one of the
> following states:
>
> * Verified - The erratum is appropriate under the criteria below and should be available to
>   implementers or people deploying the RFC.
> * Rejected - The erratum is invalid or proposes a significant change to the RFC that
>   should be done by publishing a new RFC that replaces or updates the current one. In
>   the latter case, if the change is to be considered for future updates of the document, it
>   should be proposed using channels other than the errata process, such as a WG mailing
>   list.
> * Hold for Document Update - The erratum is not a necessary update to the RFC.
>   However, any future update of the document might consider it and determine whether it
>   merits including in an update.
>
> Guidelines for review are:
>
> 1. Grammar corrections and typographical errors should be classified as Verified.
> 2. Broken URIs that were likely valid at the time of publication are, strictly speaking, not
>    subject to errata reports.  That said, the AD must judge the importance of correcting
>    such a reference and may classify the report as Verified.
> 3. Changes that are stylistic issues or simply make things read better should be classified
>    as Hold for Document Update.
> 4. Technical items that have a clear resolution in line with the original intent should be
>    classified as Verified.  If the resolution is not clear or requires further discussion, the
>    report should be classified as Hold for Document Update.  In both cases, only items that
>    are clearly wrong should be considered.
> 5. Changes that modify the working of a protocol to something that might be different from
>    the intended consensus when the document was approved should generally be
>    Rejected.  Significant clarifications should not be handled as errata reports and need to
>    be discussed by the relevant technical community.
> 6. Changes that modify the working of a process, such as changing an IANA registration
>    procedure, to something that might be different from the intended consensus when the
>    document was approved should be Rejected.
> 7. Errata on obsolete RFCs should be considered according to whether the error persists in
>    the obsoleting RFC.  If it does, the report should Rejected with a pointer to new errata
>    against the obsoleting RFC.  If it does not, it should be Rejected with an explanation that
>    the error is corrected in the obsoleting RFC (cited by number).
>
> _______________________________________________
> IETF-Announce mailing list
> [hidden email]
> https://www.ietf.org/mailman/listinfo/ietf-announce
>

Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Eric Vyncke (evyncke)
Brian

Just replying as myself and not in the name of the IESG. I tend to process all incoming errata immediately and on less loaded weeks (no formal telechat), then I try to process a couple of errata (sometimes on very very old documents).

Sometimes, I feel like I am a Danaïdes' brother __

-éric

-----Original Message-----
From: ietf <[hidden email]> on behalf of Brian E Carpenter <[hidden email]>
Date: Saturday, 8 May 2021 at 04:30
To: IETF discussion list <[hidden email]>
Subject: Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

    Hi,

    Does the IESG plan to catch up on old reported errata that have never been processed?

    There are three here for example: https://www.rfc-editor.org/errata/rfc6275, as much as 4 years old. There may be a lot more lurking.

    Regards
       Brian Carpenter

    On 08-May-21 04:38, IESG Secretary wrote:
    > IESG Processing of RFC Errata for the IETF Stream
    >
    > Online: <https://www.ietf.org/about/groups/iesg/statements/processing-errata-ietf-stream/>
    >
    > This document describes how the IESG processes RFC Errata for the IETF Stream.
    >
    > These are strong guidelines, not immutable rules. The Area Directors (ADs) should use
    > common sense and good judgment to decide what the right thing to do is. They apply to new
    > errata and not to errata that have already been processed.
    >
    > Errata are meant to fix "bugs" in the specification and should not be used to change what the
    > community meant when it approved the RFC.  Here are some things to consider when
    > submitting an errata report:
    >
    > * Errata are items that were errors at the time the document was published -- things that
    >   were missed during the last call, approval, and publication process.  If new information,
    >   new capabilities, or new thinking has come up since publication, or if you disagree with
    >   the content of the RFC, that is not material for an errata report.  Such items are better
    >   brought to relevant working groups, technical area discussions, or the IESG.
    > * Errata reports are usually for errors in the text version of a document.  It is possible to
    >   report errors in other outputs (e.g., HTML or PDF) for RFCs published in the v3 format
    >   (i.e., RFC 8650+).
    > * Errata are classified as "technical" or "editorial".  Please mark the report appropriately.  
    >   Technical errata are expected to be things that would likely cause significant
    >   misunderstandings of the technical specification and might result in faulty
    >   implementations if they are not corrected. Editorial errata are, as the name implies,
    >   editorial - for example, typos, missing commas, etc. Errors in examples will generally be
    >   editorial, though marking them as technical could sometimes be justified.
    > * Please clearly explain the issue in the Comments section. This is especially important for
    >   editorial issues, where the Original Text and Corrected Text may look almost identical.
    >
    > When a technical erratum is reported, a report is sent to the authors, chairs, and Area Directors
    > (ADs) of the WG in which the document originated. If the WG has closed or the document was
    > not associated with a WG, then the report will be sent to the ADs for the Area most closely
    > associated with the subject matter.  
    >
    > When an editorial erratum is reported, the RFC Editor will do an initial review and handle errata
    > that are clearly editorial in nature. If the erratum cannot be handled by the RFC Editor, the AD
    > will be asked to review.
    >
    > While ADs are ultimately responsible for processing the reports, they may delegate the review
    > or perform it personally.  The reviewer will classify the erratum as falling under one of the
    > following states:
    >
    > * Verified - The erratum is appropriate under the criteria below and should be available to
    >   implementers or people deploying the RFC.
    > * Rejected - The erratum is invalid or proposes a significant change to the RFC that
    >   should be done by publishing a new RFC that replaces or updates the current one. In
    >   the latter case, if the change is to be considered for future updates of the document, it
    >   should be proposed using channels other than the errata process, such as a WG mailing
    >   list.
    > * Hold for Document Update - The erratum is not a necessary update to the RFC.
    >   However, any future update of the document might consider it and determine whether it
    >   merits including in an update.
    >
    > Guidelines for review are:
    >
    > 1. Grammar corrections and typographical errors should be classified as Verified.
    > 2. Broken URIs that were likely valid at the time of publication are, strictly speaking, not
    >    subject to errata reports.  That said, the AD must judge the importance of correcting
    >    such a reference and may classify the report as Verified.
    > 3. Changes that are stylistic issues or simply make things read better should be classified
    >    as Hold for Document Update.
    > 4. Technical items that have a clear resolution in line with the original intent should be
    >    classified as Verified.  If the resolution is not clear or requires further discussion, the
    >    report should be classified as Hold for Document Update.  In both cases, only items that
    >    are clearly wrong should be considered.
    > 5. Changes that modify the working of a protocol to something that might be different from
    >    the intended consensus when the document was approved should generally be
    >    Rejected.  Significant clarifications should not be handled as errata reports and need to
    >    be discussed by the relevant technical community.
    > 6. Changes that modify the working of a process, such as changing an IANA registration
    >    procedure, to something that might be different from the intended consensus when the
    >    document was approved should be Rejected.
    > 7. Errata on obsolete RFCs should be considered according to whether the error persists in
    >    the obsoleting RFC.  If it does, the report should Rejected with a pointer to new errata
    >    against the obsoleting RFC.  If it does not, it should be Rejected with an explanation that
    >    the error is corrected in the obsoleting RFC (cited by number).
    >
    > _______________________________________________
    > IETF-Announce mailing list
    > [hidden email]
    > https://www.ietf.org/mailman/listinfo/ietf-announce
    >


Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Michael Richardson-11
In reply to this post by Brian E Carpenter-2

Brian E Carpenter <[hidden email]> wrote:
    > Does the IESG plan to catch up on old reported errata that have never been processed?

    > There are three here for example:
    > https://www.rfc-editor.org/errata/rfc6275, as much as 4 years
    > old. There may be a lot more lurking.

We need to fix the tooling to delegate to WG chairs to propose actions.
Maybe we want ADs to confirm (like milestones), but I don't think we'll ever
deal with backlog until we can easily keep up with current efforts.

--
Michael Richardson <[hidden email]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide





signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Brian E Carpenter-2
On 09-May-21 01:54, Michael Richardson wrote:
>
> Brian E Carpenter <[hidden email]> wrote:
>     > Does the IESG plan to catch up on old reported errata that have never been processed?
>
>     > There are three here for example:
>     > https://www.rfc-editor.org/errata/rfc6275, as much as 4 years
>     > old. There may be a lot more lurking.
>
> We need to fix the tooling to delegate to WG chairs to propose actions.

Yes, except when the WG no longer exists.

> Maybe we want ADs to confirm (like milestones), but I don't think we'll ever
> deal with backlog until we can easily keep up with current efforts.

I'm sure that's true too.

Dare I suggest monthly nag messages, to the WG chairs if they exist, or to the responsible AD otherwise? Or a least, a page that lists all unprocessed errata and their age in days. At the moment I think we don't even know the size of the backlog.

    Brian

Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Bob Hinden-3
In reply to this post by Brian E Carpenter-2
IESG,

I think this is good and appreciate the IESG publishing it.

There is an issue that this does not cover, that something needs to be done about.  It is when an Errata if filed, it identify the problem the Errata is addressing, and new includes text to fix the problem.

However, we have run into errata where the problem identified is correct, but the the fix to the problem is wrong.   It may be completely wrong, or there may be a better way to fix the problem.  In the worst case, it could make the problem worse.

The three states for processing an errata are:

* Verified
* Rejected
* Hold for Document Update

These don’t address this issue.   For example, marking the errata as “Verified” is fine for the problem, but not good for the fix in the errata.    We wouldn’t want implementors to assume the fix is correct.

I think something is needed where the reported problem can be accepted, but the fix can be rejected.    Perhaps some new states, or a change to how the Errata system works.

Bob


> On May 7, 2021, at 7:30 PM, Brian E Carpenter <[hidden email]> wrote:
>
> Hi,
>
> Does the IESG plan to catch up on old reported errata that have never been processed?
>
> There are three here for example: https://www.rfc-editor.org/errata/rfc6275, as much as 4 years old. There may be a lot more lurking.
>
> Regards
>   Brian Carpenter
>
> On 08-May-21 04:38, IESG Secretary wrote:
>> IESG Processing of RFC Errata for the IETF Stream
>>
>> Online: <https://www.ietf.org/about/groups/iesg/statements/processing-errata-ietf-stream/>
>>
>> This document describes how the IESG processes RFC Errata for the IETF Stream.
>>
>> These are strong guidelines, not immutable rules. The Area Directors (ADs) should use
>> common sense and good judgment to decide what the right thing to do is. They apply to new
>> errata and not to errata that have already been processed.
>>
>> Errata are meant to fix "bugs" in the specification and should not be used to change what the
>> community meant when it approved the RFC.  Here are some things to consider when
>> submitting an errata report:
>>
>> * Errata are items that were errors at the time the document was published -- things that
>>  were missed during the last call, approval, and publication process.  If new information,
>>  new capabilities, or new thinking has come up since publication, or if you disagree with
>>  the content of the RFC, that is not material for an errata report.  Such items are better
>>  brought to relevant working groups, technical area discussions, or the IESG.
>> * Errata reports are usually for errors in the text version of a document.  It is possible to
>>  report errors in other outputs (e.g., HTML or PDF) for RFCs published in the v3 format
>>  (i.e., RFC 8650+).
>> * Errata are classified as "technical" or "editorial".  Please mark the report appropriately.
>>  Technical errata are expected to be things that would likely cause significant
>>  misunderstandings of the technical specification and might result in faulty
>>  implementations if they are not corrected. Editorial errata are, as the name implies,
>>  editorial - for example, typos, missing commas, etc. Errors in examples will generally be
>>  editorial, though marking them as technical could sometimes be justified.
>> * Please clearly explain the issue in the Comments section. This is especially important for
>>  editorial issues, where the Original Text and Corrected Text may look almost identical.
>>
>> When a technical erratum is reported, a report is sent to the authors, chairs, and Area Directors
>> (ADs) of the WG in which the document originated. If the WG has closed or the document was
>> not associated with a WG, then the report will be sent to the ADs for the Area most closely
>> associated with the subject matter.
>>
>> When an editorial erratum is reported, the RFC Editor will do an initial review and handle errata
>> that are clearly editorial in nature. If the erratum cannot be handled by the RFC Editor, the AD
>> will be asked to review.
>>
>> While ADs are ultimately responsible for processing the reports, they may delegate the review
>> or perform it personally.  The reviewer will classify the erratum as falling under one of the
>> following states:
>>
>> * Verified - The erratum is appropriate under the criteria below and should be available to
>>  implementers or people deploying the RFC.
>> * Rejected - The erratum is invalid or proposes a significant change to the RFC that
>>  should be done by publishing a new RFC that replaces or updates the current one. In
>>  the latter case, if the change is to be considered for future updates of the document, it
>>  should be proposed using channels other than the errata process, such as a WG mailing
>>  list.
>> * Hold for Document Update - The erratum is not a necessary update to the RFC.
>>  However, any future update of the document might consider it and determine whether it
>>  merits including in an update.
>>
>> Guidelines for review are:
>>
>> 1. Grammar corrections and typographical errors should be classified as Verified.
>> 2. Broken URIs that were likely valid at the time of publication are, strictly speaking, not
>>   subject to errata reports.  That said, the AD must judge the importance of correcting
>>   such a reference and may classify the report as Verified.
>> 3. Changes that are stylistic issues or simply make things read better should be classified
>>   as Hold for Document Update.
>> 4. Technical items that have a clear resolution in line with the original intent should be
>>   classified as Verified.  If the resolution is not clear or requires further discussion, the
>>   report should be classified as Hold for Document Update.  In both cases, only items that
>>   are clearly wrong should be considered.
>> 5. Changes that modify the working of a protocol to something that might be different from
>>   the intended consensus when the document was approved should generally be
>>   Rejected.  Significant clarifications should not be handled as errata reports and need to
>>   be discussed by the relevant technical community.
>> 6. Changes that modify the working of a process, such as changing an IANA registration
>>   procedure, to something that might be different from the intended consensus when the
>>   document was approved should be Rejected.
>> 7. Errata on obsolete RFCs should be considered according to whether the error persists in
>>   the obsoleting RFC.  If it does, the report should Rejected with a pointer to new errata
>>   against the obsoleting RFC.  If it does not, it should be Rejected with an explanation that
>>   the error is corrected in the obsoleting RFC (cited by number).
>>
>> _______________________________________________
>> IETF-Announce mailing list
>> [hidden email]
>> https://www.ietf.org/mailman/listinfo/ietf-announce
>>
>


signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Stewart Bryant-2
In reply to this post by Brian E Carpenter-2

I'm sure that's true too.

Dare I suggest monthly nag messages, to the WG chairs if they exist, or to the responsible AD otherwise? Or a least, a page that lists all unprocessed errata and their age in days. At the moment I think we don't even know the size of the backlog.


You can find the number by area and status here:


Not sure if there is a consolidated view that shows them all, but it is easy to determine the number of unprocessed errata (status = reported) for each area.

Responsibility for moving them forward rests with the AD, and not the WG chairs, though WG have a role to play in their resolution. However in many cases the resolution is obvious and can be done quite quickly by the AD.

Perhaps we should require the IESG Chair to report the errata stats by area at each IETF?

- Stewart






Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Spencer Dawkins at IETF
In reply to this post by Michael Richardson-11
A bit more than +1 here ... 

On Sat, May 8, 2021 at 8:55 AM Michael Richardson <[hidden email]> wrote:

Brian E Carpenter <[hidden email]> wrote:
    > Does the IESG plan to catch up on old reported errata that have never been processed?

    > There are three here for example:
    > https://www.rfc-editor.org/errata/rfc6275, as much as 4 years
    > old. There may be a lot more lurking.

Can I just thank Brian for not picking something from TSV? 

Martin and I reeked at that, although we were making progress after a year or two ... Mirja inspired me to Do Better, but I don't remember us getting to zero. Based on https://www.rfc-editor.org/errata_search.php?rec_status=2&area_acronym=tsv&errata_type=2&presentation=table including one errata reported by Alfred Hoenes in 2010, I'm guessing we never did. 

But this informs my background below.
 
We need to fix the tooling to delegate to WG chairs to propose actions.
Maybe we want ADs to confirm (like milestones), but I don't think we'll ever
deal with backlog until we can easily keep up with current efforts.

This is my +1 for Michael, but I want to provide a bit of background here. 

There is a pretty big range of topics in a single area, between areas. In TSV, I could pretty trivially process technical errata for some working groups, but for other working groups, I had paid almost no attention to them for years, before becoming AD (I love NFSv4, but I was only vaguely aware that NFS was now running over TCP). 

I am pretty sure that there are some serving ADs who are responsible for working groups that they've never followed and had to subscribe to the mailing lists when they were seated, and had never even looked at those working group charters (especially not the historical charters). I suspect that's more true in some areas than others, but that's just a guess. 

As long as working groups exist, it's likely that WG chairs have fewer errata to look at than ADs, and also that they are more likely to be familiar with the documents and topics from their own working groups. 

So, yes, PLEASE consider doing with errata what has been done with milestones. That would make a huge difference for at least some errata from current working groups, and if the ADs end up dealing with the deep end of the swamp without an active working group (or even an active mailing list), well, that's just part of the job. Today, they own all of the swamp.  

This is much more likely to work well than nag messages (ADs have some of the most fabulous mail filters I've ever seen, just in order to survive, and once you figure out you won't be recalled if there's a 10-year-old reported errata in your area, there's no going back!).

Best,

Spencer
 
--
Michael Richardson <[hidden email]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide
Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Spencer Dawkins at IETF
In reply to this post by Stewart Bryant-2
Hi, Stewart, 

On Sun, May 9, 2021 at 3:29 AM Stewart Bryant <[hidden email]> wrote:

Perhaps we should require the IESG Chair to report the errata stats by area at each IETF?

We might disagree on how responsive ADs have been, or should be, to public shaming in general, but I remember that the errata states were presented at plenary when I joined the IESG, and Martin and I managed to be pretty shameless, when it came to errata 😂


Best,

Spencer
Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Carsten Bormann
In reply to this post by Spencer Dawkins at IETF
On 2021-05-09, at 20:54, Spencer Dawkins at IETF <[hidden email]> wrote:
>
>  https://www.rfc-editor.org/errata_search.php?rec_status=2&area_acronym=tsv&errata_type=2&presentation=table

Wow, thanks.

I immediately glanced at

https://www.rfc-editor.org/errata_search.php?rec_status=2&area_acronym=art&errata_type=2&presentation=table

228 of them.  Two even reported by me that I had completely forgotten about :*)

So how do I find unhandled errata reports for all RFCs that I’m a co-author on?

Grüße, Carsten

:*) https://www.rfc-editor.org/errata_search.php?rec_status=2&errata_type=2&presentation=table&submitter_name=Carsten%20Bormann

Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

tom petch-3
In reply to this post by Bob Hinden-3
On 08/05/2021 23:53, Bob Hinden wrote:

> IESG,
>
> I think this is good and appreciate the IESG publishing it.
>
> There is an issue that this does not cover, that something needs to be done about.  It is when an Errata if filed, it identify the problem the Errata is addressing, and new includes text to fix the problem.
>
> However, we have run into errata where the problem identified is correct, but the the fix to the problem is wrong.   It may be completely wrong, or there may be a better way to fix the problem.  In the worst case, it could make the problem worse.
>
> The three states for processing an errata are:
>
> * Verified
> * Rejected
> * Hold for Document Update
>
> These don’t address this issue.   For example, marking the errata as “Verified” is fine for the problem, but not good for the fix in the errata.    We wouldn’t want implementors to assume the fix is correct.
>
> I think something is needed where the reported problem can be accepted, but the fix can be rejected.    Perhaps some new states, or a change to how the Errata system works.

I have seen this several times and the solution I see is that the AD has
space to comment on the reasons for the rejection and that is what they
do, yes to the problem, no to the solution, and in several cases the
author of the erratum has another go, perhaps with a WG discussion, and
in the end there is usually an accepted erratum.

Better still is to have the discussion on the WG list beforehand and I
see plenty of that.  In fact, I would say that an erratum without a
prior discussion usually comes from someone whom I have not seen
participating in the IETF before, is unfamiliar with the process and do
not realise that functional changes are made via an I-D.

Tom Petch
Tom Petch

> Bob
>
>
>> On May 7, 2021, at 7:30 PM, Brian E Carpenter <[hidden email]> wrote:
>>
>> Hi,
>>
>> Does the IESG plan to catch up on old reported errata that have never been processed?
>>
>> There are three here for example: https://www.rfc-editor.org/errata/rfc6275, as much as 4 years old. There may be a lot more lurking.
>>
>> Regards
>>    Brian Carpenter
>>
>> On 08-May-21 04:38, IESG Secretary wrote:
>>> IESG Processing of RFC Errata for the IETF Stream
>>>
>>> Online: <https://www.ietf.org/about/groups/iesg/statements/processing-errata-ietf-stream/>
>>>
>>> This document describes how the IESG processes RFC Errata for the IETF Stream.
>>>
>>> These are strong guidelines, not immutable rules. The Area Directors (ADs) should use
>>> common sense and good judgment to decide what the right thing to do is. They apply to new
>>> errata and not to errata that have already been processed.
>>>
>>> Errata are meant to fix "bugs" in the specification and should not be used to change what the
>>> community meant when it approved the RFC.  Here are some things to consider when
>>> submitting an errata report:
>>>
>>> * Errata are items that were errors at the time the document was published -- things that
>>>   were missed during the last call, approval, and publication process.  If new information,
>>>   new capabilities, or new thinking has come up since publication, or if you disagree with
>>>   the content of the RFC, that is not material for an errata report.  Such items are better
>>>   brought to relevant working groups, technical area discussions, or the IESG.
>>> * Errata reports are usually for errors in the text version of a document.  It is possible to
>>>   report errors in other outputs (e.g., HTML or PDF) for RFCs published in the v3 format
>>>   (i.e., RFC 8650+).
>>> * Errata are classified as "technical" or "editorial".  Please mark the report appropriately.
>>>   Technical errata are expected to be things that would likely cause significant
>>>   misunderstandings of the technical specification and might result in faulty
>>>   implementations if they are not corrected. Editorial errata are, as the name implies,
>>>   editorial - for example, typos, missing commas, etc. Errors in examples will generally be
>>>   editorial, though marking them as technical could sometimes be justified.
>>> * Please clearly explain the issue in the Comments section. This is especially important for
>>>   editorial issues, where the Original Text and Corrected Text may look almost identical.
>>>
>>> When a technical erratum is reported, a report is sent to the authors, chairs, and Area Directors
>>> (ADs) of the WG in which the document originated. If the WG has closed or the document was
>>> not associated with a WG, then the report will be sent to the ADs for the Area most closely
>>> associated with the subject matter.
>>>
>>> When an editorial erratum is reported, the RFC Editor will do an initial review and handle errata
>>> that are clearly editorial in nature. If the erratum cannot be handled by the RFC Editor, the AD
>>> will be asked to review.
>>>
>>> While ADs are ultimately responsible for processing the reports, they may delegate the review
>>> or perform it personally.  The reviewer will classify the erratum as falling under one of the
>>> following states:
>>>
>>> * Verified - The erratum is appropriate under the criteria below and should be available to
>>>   implementers or people deploying the RFC.
>>> * Rejected - The erratum is invalid or proposes a significant change to the RFC that
>>>   should be done by publishing a new RFC that replaces or updates the current one. In
>>>   the latter case, if the change is to be considered for future updates of the document, it
>>>   should be proposed using channels other than the errata process, such as a WG mailing
>>>   list.
>>> * Hold for Document Update - The erratum is not a necessary update to the RFC.
>>>   However, any future update of the document might consider it and determine whether it
>>>   merits including in an update.
>>>
>>> Guidelines for review are:
>>>
>>> 1. Grammar corrections and typographical errors should be classified as Verified.
>>> 2. Broken URIs that were likely valid at the time of publication are, strictly speaking, not
>>>    subject to errata reports.  That said, the AD must judge the importance of correcting
>>>    such a reference and may classify the report as Verified.
>>> 3. Changes that are stylistic issues or simply make things read better should be classified
>>>    as Hold for Document Update.
>>> 4. Technical items that have a clear resolution in line with the original intent should be
>>>    classified as Verified.  If the resolution is not clear or requires further discussion, the
>>>    report should be classified as Hold for Document Update.  In both cases, only items that
>>>    are clearly wrong should be considered.
>>> 5. Changes that modify the working of a protocol to something that might be different from
>>>    the intended consensus when the document was approved should generally be
>>>    Rejected.  Significant clarifications should not be handled as errata reports and need to
>>>    be discussed by the relevant technical community.
>>> 6. Changes that modify the working of a process, such as changing an IANA registration
>>>    procedure, to something that might be different from the intended consensus when the
>>>    document was approved should be Rejected.
>>> 7. Errata on obsolete RFCs should be considered according to whether the error persists in
>>>    the obsoleting RFC.  If it does, the report should Rejected with a pointer to new errata
>>>    against the obsoleting RFC.  If it does not, it should be Rejected with an explanation that
>>>    the error is corrected in the obsoleting RFC (cited by number).
>>>
>>> _______________________________________________
>>> IETF-Announce mailing list
>>> [hidden email]
>>> https://www.ietf.org/mailman/listinfo/ietf-announce
>>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

John C Klensin
In reply to this post by Bob Hinden-3


--On Saturday, May 8, 2021 15:53 -0700 Bob Hinden
<[hidden email]> wrote:

> IESG,
>
> I think this is good and appreciate the IESG publishing it.
>
> There is an issue that this does not cover, that something
> needs to be done about.  It is when an Errata if filed, it
> identify the problem the Errata is addressing, and new
> includes text to fix the problem.
>
> However, we have run into errata where the problem identified
> is correct, but the the fix to the problem is wrong.   It may
> be completely wrong, or there may be a better way to fix the
> problem.  In the worst case, it could make the problem worse.
>
> The three states for processing an errata are:
>
> * Verified
> * Rejected
> * Hold for Document Update
>
> These don't address this issue.   For example, marking the
> errata as "Verified" is fine for the problem, but not good
> for the fix in the errata.    We wouldn't want implementors
> to assume the fix is correct.
>
> I think something is needed where the reported problem can be
> accepted, but the fix can be rejected.    Perhaps some new
> states, or a change to how the Errata system works.

Bob,

While I agree with your statement of the problem, I'm not sure
about the fix :-).  And I'm afraid you just opened a can of
worms.

Let's take the two cases I think are difficult and putting aside
documents that are not on the standards track or that did not
come from a WG for the moment:

Example 1:  Text says "...Mickey is a duck".  Errata report says
'"not" is missing; Mickey is not a duck' and claims it is an
editorial matter.

Now, I would hope the RFC Editor would say "the suggestion
changes the meaning of the sentence and is consequently not
editorial" but I would hope there would be an additional set of
eyes on that decision, or any decision that something more
serious than, e.g., an obvious misspelling, is editorial.  To
reduce AD  workload, I think those eyes can safely be those of
the author(s).

Example 2: There is a real technical issue.  The assertion about
Mickey's species might turn out to be an example but I'm more
concerned about proposed errata that involve more complex
issues.  If the consensus among ADs, Authors, and present or
former WG Chairs is "whoops, the WG didn't think about that and
thinking about that now gives us a headache", then the erratum
clearly gets "save for document revision" and we move on.  But
suppose it is "Verified".  What does that mean and, coming back
to your point in particular, can an implementer rely on it?  It
does not represent IETF Consensus: there is been no IETF Last
Call and it is really the position of a handful of people.

My guess is that, at least for technical issues, "hold for
document update"  means "we are not going to think about that
now" and that "verified" means "we agree that there is probably
a problem, the proposed fix may or may not be correct, and you
will need to wait for a new RFC with IETF Consensus to get a
conclusion implementers can rely on".  The IESG should be able
to make statements about the level of their confidence in the
fix as long as it was clear that their comments must not be
confused with IETF consensus either.  That would, of course,
solve the problem you identify but has much broader
implications.  If there is agreement about "verified" not
representing IETF consensus -- even for the problem statement,
much less the fix-- I'd hope the relevant RFC Editor pages could
be updated to clearly show definitions like those and that IESG
statements could point to them.

These distinctions are particularly important if the IETF is
going to publish versions of RFCs with errata incorporated.

The only other option I can see involves putting every single
erratum with potentially substantive technical implications
through IETF Last Call before it is marked "verified".  Even if
that was not required for "rejected" or "save for document
update", to say that would not reduce AD (or community) workload
significantly would be an understatement.  

best,
   john

Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Spencer Dawkins at IETF
Massively chopping this thread of (I'm not being snarky) wisdom, down to the part I think I can contribute to ...

On Sun, May 9, 2021 at 3:20 PM John C Klensin <[hidden email]> wrote:

These distinctions are particularly important if the IETF is
going to publish versions of RFCs with errata incorporated.

Because I honestly don't know who might be reading this e-mail thread, and what they might know, let me say this out loud.

I have no inside knowledge of what the *IETF* might plan to publish, but for what the RFC Editor publishes, this ship has sailed. 

Tl;dr 

I had the privilege (HAH!!!) of explaining the Theory and Practice of RFC Errata to members of the CELLAR working group that I co-chair with Michael Richardson, during our last virtual meeting. This group has published one RFC, and https://datatracker.ietf.org/doc/html/rfc8794 is 2700 lines long and pretty dense, so they're receiving errata report (mostly through Github, from people who don't even participate in CELLAR, much less grok the difference between the RFC view in the datatracker and the RFC view on rfc-editor.org).

They were horrified to discover that RFCs are immutable, and were imagining that they were going to have to roll another RFC every time someone reported a bug. 

I showed them (as an example) that if they searched for RFC3261 on the RFC Editor website, and clicked on the cute "HTML with a green check mark" icon, they got a page that started with this:

This is a purely informative rendering of an RFC that includes verified errata. This rendering may not be used as a reference.

The following 'Verified' errata have been incorporated in this document: EID 316EID 1051EID 1073EID 1470EID 2051EID 3183EID 4567


And if you click on the links, it shows you where each errata goes, and if you click on "Expand", it shows you the Verified errata, and if the errata had OLD/NEW text, the new text is shown inline. 

So, what you get is something like this:

   The handling parameter, handling-param, describes how the UAS should 
react if it receives a message body whose content type or disposition
type it does not understand. The parameter has defined values of
"optional" and "required". If the handling parameter is missing, or if
the Content-Disposition header field is missing, the value "required" 
SHOULD be assumed. The handling parameter is described in RFC 3204 [19].

EID 4567 (Verified) is as follows:

Section: 20.11

Original Text:

The handling parameter, handling-param, describes how the UAS should
react if it receives a message body whose content type or disposition
type it does not understand. The parameter has defined values of
"optional" and "required". If the handling parameter is missing, the
value "required" SHOULD be assumed. The handling parameter is
described in RFC 3204 [19].

Corrected Text:

The handling parameter, handling-param, describes how the UAS should
react if it receives a message body whose content type or disposition
type it does not understand. The parameter has defined values of
"optional" and "required". If the handling parameter is missing, or if
the Content-Disposition header field is missing, the value "required" 
SHOULD be assumed. The handling parameter is described in RFC 3204 [19].
Notes:
None

So, they're SUPER excited about making use of this mechanism, and we can talk about whether people really read Public Service Announcements ("This is a purely informative rendering of an RFC that includes verified errata. This rendering may not be used as a reference.") - but can we blame them for wanting to do that?

So, ISTM that what our current errata system was set up to do (and I can't emphasize enough that I wasn't there when it was set up), was to Reject stuff that wasn't REALLY an errata or obviously not the right correction to make, Verify obvious corrections, and Hold Everything Else For Document Updates, so that a working group has a chance to talk about what The Right Thing To Do is, given that they missed something Technical when they looked at this for the first time, as opposed to having an AD decide *now* what The Right Thing To Do will be at some point in the future, and constraining the working group when they *do* have enough brain cells to update the document. (If that's not the case, my apologies to the people who set it up in the first place). 

If we wanted to make life easier for people using IETF specifications (as opposed to going to the Github repos and "making" their own specifications from Master/Main branches, we might think about
  • How much we want to "warn people away" from using the really helpful errata inlining mechanism that is already available from the RFC Editor, and 
  • How much stuff that's not Verified would be useful to present to the people who use our documents - especially stuff that's Rejected because it doesn't reflect the thinking of (my own favorite) the NFSv4 working group more than a decade ago, but is really close to what they are thinking today.
As always, thanks for listening. And Do The Right Thing. 

Best,

Spencer, who used the tools format of RFCs in HTML almost exclusively for at least a decade, and then switched to the datatracker HTML format, neither of which were the canonical txt-then, XML-now formats that we wish people would use. 
Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

otroan
In reply to this post by Bob Hinden-3
> IESG,

>
> I think this is good and appreciate the IESG publishing it.
>
> There is an issue that this does not cover, that something needs to be done about.  It is when an Errata if filed, it identify the problem the Errata is addressing, and new includes text to fix the problem.
>
> However, we have run into errata where the problem identified is correct, but the the fix to the problem is wrong.   It may be completely wrong, or there may be a better way to fix the problem.  In the worst case, it could make the problem worse.
>
> The three states for processing an errata are:
>
> * Verified
> * Rejected
> * Hold for Document Update
>
> These don’t address this issue.   For example, marking the errata as “Verified” is fine for the problem, but not good for the fix in the errata.    We wouldn’t want implementors to assume the fix is correct.
>
> I think something is needed where the reported problem can be accepted, but the fix can be rejected.    Perhaps some new states, or a change to how the Errata system works.
I fully agree with that.
I have also seen cases where errata is wrong or even has been used as a way to try to change a specifictation.

The current errata system allows anyone to attach whatever they like without review to a published document.
I don't know how to fix this, but it might include changes to process, tools and presentation.
A clearer distinction between a reported document bug and a verified errata could be a start.

Best regards,
Ole

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Carsten Bormann
On 2021-05-10, at 08:31, [hidden email] wrote:
>
> A clearer distinction between a reported document bug and a verified errata could be a start.

Yes.  

(1) We need to stop calling errata reports that are not (yet) verified, “errata”.

The errata are the problems that we *agree* exist.
(Each single such problem is an erratum, BTW.)

Section 4.8.6.5 of RFC 7322 MUST die, die, die.

(My own errata report about the incorrect terminology of that section was almost rejected…  https://www.rfc-editor.org/errata/eid6347 (*))

(2) When an errata report is indeed verified, it should be OPTIONAL whether the suggested solution is also verified.  I’m not unhappy about the idea of last-calling an actual solution instead, after it has been discussed and shaped by authors and the WG (or the WG’s old mailing list, or a dispatch WG if that doesn’t exist).

Grüße, Carsten

(*)  [Err4409] [Err4963] [Err4964] were attempts to change the (admittedly not so bright, after all) WG consensus after the fact.  They are still cited as “errata” in RFC 8949 in Appendix G, where we discuss how we handled the errata reports on its predecessor RFC 7049.  Reason did not prevail over Section 4.8.6.5.

Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Lloyd W
In reply to this post by otroan
if we're changing how this works...

the singular of errata is erratum

one erratum, many errata
one criterion, many criteria
one datum, many data

one didactic pedant, many ...

sigh

Lloyd Wood
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

John C Klensin
In reply to this post by Brian E Carpenter-2
Spencer,

Trimming further -- your explanation about what has gone on / is
going on in the CELLAR WG was interesting and helpful, but I
have little or nothing to add that is specific to it.  On the
other hand, a situation where there is an active WG that
developed the document on which errata were filed seems to me
much different -- and with far more opportunity for real
consensus about decisions-- than the more usual errata
situations.  See below.


--On Sunday, May 9, 2021 16:45 -0500 Spencer Dawkins at IETF
<[hidden email]> wrote:

>...
> So, ISTM that what our current errata system was set up to do
> (and I can't emphasize enough that I wasn't there when it was
> set up), was to Reject stuff that wasn't REALLY an errata or
> obviously not the right correction to make, Verify obvious
> corrections, and Hold Everything Else For Document Updates, so
> that a working group has a chance to talk about what The Right
> Thing To Do is, given that they missed something Technical
> when they looked at this for the first time, as opposed to
> having an AD decide *now* what The Right Thing To Do will be
> at some point in the future, and constraining the working
> group when they *do* have enough brain cells to update the
> document. (If that's not the case, my apologies to the people
> who set it up in the first place).

But that view suggests that Bob's case of a proposed erratum
correctly identifying a problem but not the right fix should
lead to a Reject conclusion, and that does not seem quite right
either.

> If we wanted to make life easier for people using IETF
> specifications (as opposed to going to the Github repos and
> "making" their own specifications from Master/Main branches,
> we might think about
>
>    - How much we want to "warn people away" from using the
> really helpful    errata inlining mechanism that is already
> available from the RFC Editor,    and
>    - How much stuff that's not Verified would be useful to
> present to the    people who use our documents - especially
> stuff that's Rejected because it    doesn't reflect the
> thinking of (my own favorite) the NFSv4 working group    more
> than a decade ago, but is really close to what they are
> thinking today.
>
> As always, thanks for listening. And Do The Right Thing.

For a situation where there is an active WG with the best
document as part of its charter, let me suggest something else,
starting with wondering whether the errata system is the right
approach at all.  If, as I gather is the case you are talking
about, the WG gets a document published and then immediately
starts discovering bugs in it, isn't The Right Thing for the WG
to put a revision onto its agenda, get a new I-D started, and
put the corrections there?  THere makes the status immediately
clear as "WG work in progress" without any handwaving about what
"Verified" actually means.  A "Changes from RFC NNNN" section,
which would be desirable anyway, could explain things to the
degree desired.  If the WG wanted to put a note on that I-D that
says that everything in it represents WG consensus, that would
be much more clear than the present errata process [1].

The missing link is something that might be desirable for many
other reasons: It seems to me that, if an RFC is actively under
revision in an IETF WG, an "under revision" note on the RFC
Editor's page for that RFC with a pointer to the WG charter
and/or the relevant I-D would be a big help to readers.  It
would also capture the common case in which WGs make many
changes to a document on which errata are not filed.  

However successful we are (or are not) at convincing people that
I-Ds are not standards, pointing a reader to a revision in
progress makes it clear that something is going on, may give
insights into the situation with the existing RFC. In the best
of all possible worlds, it might even encourage people who think
they are finding problems in a document to participate in the
relevant WG and discuss their issues.

Finally, if issues are addressed in a WG that is active and
responsible for a document anyway, that reduces the workload on
everyone who would need to be part of a round trip through the
errata system before the issue gets back to that WG anyway.

With the understanding that the situation in particular WGs may
be different, does that sound to you like possibly a better
approximation to The Right Thing for many or most cases?

   best,
    john


[1] While I would hope ADs or Chairs of active WGs with
responsibility for a document against errata were filed would
consult with the WG about the erratum, there is nothing in the
IESG statement that requires that.

Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Michael Richardson-11
In reply to this post by Carsten Bormann

Carsten Bormann <[hidden email]> wrote:
    > So how do I find unhandled errata reports for all RFCs that I’m a co-author on?

Ultimately, that's the workflow that we need to enable.
It's not just the finding, it's getting the response into a clear place so
that the AD can easily digest and approve without spending multiple hours
per.

--
Michael Richardson <[hidden email]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide





signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Alvaro Retana-2
In reply to this post by Bob Hinden-3
On May 8, 2021 at 6:53:46 PM, Bob Hinden wrote:


Bob:

Hi!

...
> I think something is needed where the reported problem can be accepted, but
> the fix can be rejected. Perhaps some new states, or a change to how the
> Errata system works.

You're right, *without proper explanation*, there is no explicit state
that indicates a valid problem and an invalid solution.  Hold for
Document Update is the closest as further discussion is obviously
needed.

However, even with a proper explanation, it may still be confusing
whether the proposed solution is valid or not.  One of the issues is
that there is a single notes field that is used by both the submitter
and the verifier.  Also, these notes appear *after* the
problem/solution have been described, making it harder to find
relevant comments.

Changing how the errata system works requires a wider discussion of
course -- beyond what the current statement is intended for.

Thanks!

Alvaro.

Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Brian E Carpenter-2
In reply to this post by Carsten Bormann
On 10-May-21 07:11, Carsten Bormann wrote:
> On 2021-05-09, at 20:54, Spencer Dawkins at IETF <[hidden email]> wrote:
>>
>>  https://www.rfc-editor.org/errata_search.php?rec_status=2&area_acronym=tsv&errata_type=2&presentation=table
>
> Wow, thanks.

Yes, thanks. So we can quickly see that there are 556 unhandled errata, and the oldest ones were reported in 2010. That seems to be about 8% of the total errata ever reported. Is this a problem worth fixing?

On the entertainment side, there's one very plausible erratum on RFC1321 (MD5) reported last year, and one on RFC 1180, the "TCP/IP tutorial" from
1991, reported in 2017.

And in case anybody's wondering, I did *not* report the erratum #5209 on RFC1001. But it might well be valid.

    Brian

>
> I immediately glanced at
>
> https://www.rfc-editor.org/errata_search.php?rec_status=2&area_acronym=art&errata_type=2&presentation=table
>
> 228 of them.  Two even reported by me that I had completely forgotten about :*)
>
> So how do I find unhandled errata reports for all RFCs that I’m
a co-author on?
>
> Grüße, Carsten
>
> :*) https://www.rfc-editor.org/errata_search.php?rec_status=2&errata_type=2&presentation=table&submitter_name=Carsten%20Bormann
>

Reply | Threaded
Open this post in threaded view
|

Re: Updated IESG Statement "IESG Processing of RFC Errata for the IETF Stream"

Spencer Dawkins at IETF
In reply to this post by John C Klensin
Hi, John,

On Mon, May 10, 2021 at 7:46 AM John C Klensin <[hidden email]> wrote:
Spencer,

Trimming further -- your explanation about what has gone on / is
going on in the CELLAR WG was interesting and helpful, but I
have little or nothing to add that is specific to it.  On the
other hand, a situation where there is an active WG that
developed the document on which errata were filed seems to me
much different -- and with far more opportunity for real
consensus about decisions-- than the more usual errata
situations.  See below.


--On Sunday, May 9, 2021 16:45 -0500 Spencer Dawkins at IETF
<[hidden email]> wrote:

>...
> So, ISTM that what our current errata system was set up to do
> (and I can't emphasize enough that I wasn't there when it was
> set up), was to Reject stuff that wasn't REALLY an errata or
> obviously not the right correction to make, Verify obvious
> corrections, and Hold Everything Else For Document Updates, so
> that a working group has a chance to talk about what The Right
> Thing To Do is, given that they missed something Technical
> when they looked at this for the first time, as opposed to
> having an AD decide *now* what The Right Thing To Do will be
> at some point in the future, and constraining the working
> group when they *do* have enough brain cells to update the
> document. (If that's not the case, my apologies to the people
> who set it up in the first place).

But that view suggests that Bob's case of a proposed erratum
correctly identifying a problem but not the right fix should
lead to a Reject conclusion, and that does not seem quite right
either.

I'm sorry if I wasn't clear - the understanding I was describing was my own, in general, and definitely not talking specifically about CELLAR. 

Your point above - about an active working group making informed decisions about a document that they had produced - is very much accurate for CELLAR. Reported errors (not from the errata system, but from implementers who likely would have no idea where to go to file an errata report on an RFC) are discussed promptly on the mailing list, and if it seems like there's more than one opinion, in the next monthly meeting (CELLAR has about 10 meetings per year), and pretty quickly, they know what the answer should be. But see below. 
 
> If we wanted to make life easier for people using IETF
> specifications (as opposed to going to the Github repos and
> "making" their own specifications from Master/Main branches,
> we might think about
>
>    - How much we want to "warn people away" from using the
> really helpful    errata inlining mechanism that is already
> available from the RFC Editor,    and
>    - How much stuff that's not Verified would be useful to
> present to the    people who use our documents - especially
> stuff that's Rejected because it    doesn't reflect the
> thinking of (my own favorite) the NFSv4 working group    more
> than a decade ago, but is really close to what they are
> thinking today.
>
> As always, thanks for listening. And Do The Right Thing.

For a situation where there is an active WG with the best
document as part of its charter, let me suggest something else,
starting with wondering whether the errata system is the right
approach at all.  If, as I gather is the case you are talking
about, the WG gets a document published and then immediately
starts discovering bugs in it, isn't The Right Thing for the WG
to put a revision onto its agenda, get a new I-D started, and
put the corrections there?  THere makes the status immediately
clear as "WG work in progress" without any handwaving about what
"Verified" actually means.  A "Changes from RFC NNNN" section,
which would be desirable anyway, could explain things to the
degree desired.  If the WG wanted to put a note on that I-D that
says that everything in it represents WG consensus, that would
be much more clear than the present errata process [1].

So, let's keep solving MY problem, which isn't what to do when CELLAR gets an error report, but about what to do after they agree that they need to do something to fix the specification. 

Two things about that.

First, their RFC describes a markup language that they use to describe Matroska media containers, and their soon-to-be-PUB-REQUESTED draft describes pre-IETF versions of an archival video codec. What they really WANT to do is finish Matroska, which is (as best I can tell) their premier deliverable, and work on a new version of that archival video codec. They'd like their published specifications to be correct, but any effort they spend on corrections is taking away from effort that they can spend on their next deliverables. So, if the errata system worked the way we wish it would, that would be fine, but it just doesn't work the way we wish it would (for more reasons than I know).

So, the other thing is - I am not sure whether you are thinking of an UPDATES RFC NNNN WITH ERRATA draft, or an RFC NNNN-bis draft that replaces RFC NNNN, but each of these has its own special agony. 
  • If one were to take a look at https://datatracker.ietf.org/doc/rfc8540/ballot/, for an UPDATES RFC 4960 WITH ERRATA draft, they will  be impressed that an Informational document saying "this is what the working group thinks of its errata" managed to gather (counting again) *9* abstains from ADs who (summarizing) said roughly "no value in publishing this, Should be a wiki page. Where's the RFC 4960-bis draft?" I don't know what the current IESG thinks about docs like this, but what the 2018 IESG thought about them is well-documented(1). Note that "publishing what the working group thinks about errata in a wiki page" is exactly what CELLAR is trying to avoid, and (for extra credit) might not even cross the inbox of an AD. 
  • If one were to take a look at https://datatracker.ietf.org/doc/draft-ietf-tsvwg-rfc4960-bis/, the requested -bis draft, they would note that the UPDATES RFC 4960 WITH ERRATA draft was published almost exactly two years ago, and the REPLACES RFC 4960 draft is still sitting comfortably in the working group. 
So, we still have some work to do here. 

"We" being the IETF, and not just dumping this on the current IESG, unless they have a lot of time and energy on their hands. 

I should semi-apologize to you for forgetting that you had written https://datatracker.ietf.org/doc/html/draft-klensin-newtrk-8540style-harmful-00 on this EXACT situation, but I even commented on it at the time (in discussion on IETF mailing list). So I know you see the problem pretty clearly.   

(1) To be clear, this is NOT a knock on the 2018 IESG. I'm just pointing to a document I'm familiar with, as the then-responsible AD. 
 
The missing link is something that might be desirable for many
other reasons: It seems to me that, if an RFC is actively under
revision in an IETF WG, an "under revision" note on the RFC
Editor's page for that RFC with a pointer to the WG charter
and/or the relevant I-D would be a big help to readers.  It
would also capture the common case in which WGs make many
changes to a document on which errata are not filed. 

However successful we are (or are not) at convincing people that
I-Ds are not standards, pointing a reader to a revision in
progress makes it clear that something is going on, may give
insights into the situation with the existing RFC. In the best
of all possible worlds, it might even encourage people who think
they are finding problems in a document to participate in the
relevant WG and discuss their issues.

Finally, if issues are addressed in a WG that is active and
responsible for a document anyway, that reduces the workload on
everyone who would need to be part of a round trip through the
errata system before the issue gets back to that WG anyway.

With the understanding that the situation in particular WGs may
be different, does that sound to you like possibly a better
approximation to The Right Thing for many or most cases?

Modulo how much I wish we'd gotten https://datatracker.ietf.org/doc/html/draft-ietf-newtrk-repurposing-isd-04 out of the NEWTRK working group in 2006 because we could have stopped thinking about this more than a decade ago, yes!

Best,

Spencer
 

   best,
    john


[1] While I would hope ADs or Chairs of active WGs with
responsibility for a document against errata were filed would
consult with the WG about the erratum, there is nothing in the
IESG statement that requires that.
12