Should neutered RTCEncodedVideoFrames have type "empty"

[guidou]

The spec says
empty: This frame contains no data.

key: This frame can be decoded without reference to any other frames.

delta: This frame references another frame and can not be decoded without that frame.

IMO, this means that a transferred or written frame , which contains no data, should have type "empty".
However, we have a WPT that says it should return the same type it had before being neutered and some implementations follow this.

[youennf]

The transfer steps do not update the type of the frame so the test follows the spec.
I would tend to keep type stable over transfer, given other metadata are also stable.

I do not see any use of empty in Safari, not sure about Firefox.
Can we remove empty?
That would make RTCEncodedVideoFrameType equivalent to [EncodedVideoChunkType](https://w3c.github.io/webcodecs/#enumdef-encodedvideochunktype ?

[guidou]

There is an inconsistency in the spec because we have a [[type]] slot, but:

1. the type field is not defined to return the value of that slot.
2. the slot is never initialized to any actual value when a source produces a frame.
3. the slot is only used in the spec to be copied when using the copy constructor or when serializing.

Thus, the slot is not observable.

Therefore, I think the test is wrong. We can either change the test or change the spec, but right now the test and the spec do not agree.

[youennf]

I see the following:

• We have frame type.
• We have [[type]] for serialised data of a frame, which is used to store/set a frame type.
• We use [[type]] in the frame constructor step 1 while we should use the frame type instead.

I am fine adding a [[type]] slot for the frame, or removing the use of [[type]] in constructor.

That said, I do not see how the spec is stating that empty is the type value for transferred/written frame.

More generally, what is the value of empty?

[guidou]

The spec only says that The type attribute allows the application to determine when a key frame is being sent or received, which I think is not really the definition that we want.

The definition of the RTCEncodedVideoFrameType says "this frame can be decoded without reference to any other frames", which is true before the frame is neutered, but not after.

The [[type]] slot exists, but it's value is never set or returned. It's only used for copying or serializing frames.

With the spec text as is, probably no test can agree with it.
Since two shipped implementations remember the value of the type field after it is neutered and the test agrees with them, I'm open to changing to spec to match this behavior.

[youennf]

#279 should help a bit with the slot issues.

As of The type attribute allows the application to determine when a key frame is being sent or received, we indeed need a better definition.

There is a comment stating that we might want to replace RTCEncodedVideoFrameType with the webcodecs equivalent.
Maybe this is time to do that.