RTSP: Speed and Scale overview text

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

RTSP: Speed and Scale overview text

Magnus Westerlund
Hi,

Below is an overview text about Speed and Scale. Trying to ensure that
we all agree what Scale and Speed is supposed to do. So please review
and see if you think you understand this. If it matches your
understanding of the concepts and what need you see of either of the
mechanisms. I want to ensure that we are on the same page and do see a
need for both before updating the rest of the text regarding these subjects.

Media Delivery Manipulations

The basic playback functionality of RTSP is to request content for a
particular range to be delivered to the client in a pace that enables
playback as intended by the creator. However, RTSP can also manipulate
how this delivery is done to the client in two ways.

   Scale: The ratio of media content delivered per unit playback time.

   Speed: The ratio of playback time delivered per unit of wall clock
          time.

So both affects the media delivery per time unit. However, they are
manipulating two independent time scales and the effects are possible to
combine.

Scale is used for fast forward or slow motion control as it changes the
amount of content timescale that should be played back per time unit.
Scale > 1.0, means fast forward, e.g. Scale=2.0 results in that 2
seconds of content is played back every second of playback. Scale = 1.0
is the default value that is used if no Scale is specified, i.e.
playback at the contents original rate. Scale values between 0 and 1.0
is providing for slow motion. Scale can be negative to allow for reverse
playback in either regular pace (Scale = -1.0) or fast backwards (Scale
< -1.0) or slow motion backwards (-1.0 < Scale < 0).

In most cases the realization of scale means server side manipulation of
the media to ensure that the client can actually play it back. These
media manipulation and when they are needed are highly media type
dependent. Lets exemplify with two common media types audio and video.

It is very difficult to modify the playback rate of audio. A maximum of
10-30% is possible by changing the pitch-rate of speech. Music goes out
of tune if one tries to manipulate the playback rate by resampling it.
This is a well known problem and audio is commonly muted or played back
in short segments with skips to keep up with the current playback point.

For video is possible to manipulate the number of frames that is
displayed per second. But the rendering capabilities are often limited
to certain frame rates. The decoding, handling capabilities and bitrate
of received encoded content also limits the number of frames that can be
delivered. Therefore when providing fast forward one generally picks a
subset of the frames from the original content to be displayed. However,
the video encoding methods use will commonly limit the possibilities on
which frames that can be chosen and still be decoded by the receiver.

Due to the media restrictions a particular content will commonly be
restricted to a limited set of possible scale ratios. To handle this
correctly, RTSP has mechanism to indicate the supported Scale ratios for
the content. To support aggregated or dynamic content where this may
change during the ongoing session and dependent on the location within
the content a mechanism for updating the media properties and the
current used scale factor exist.

Speed affects how much of the playback timeline that is delivered in a
given wall clock period. The default is Speed = 1 which is to deliver at
the same rate the media is consumed. Speed > 1 means that the receiver
will get content faster than it regularly would consume it. Speed < 1
means that it receives it slower than the regular consumption rate. This
mechanism enables two general functionalities. Client side scale
operations, i.e. the client receives all the frames and makes the
adjustment to the playback locally. The second usage is to control its
buffering of media. By specifying a speed over 1.0 the client can build
up the amount of playback time it has present in its buffers to a level
that is sufficient for its needs.
--

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Burklin Helmut
Hallo,
I think for the layman the text will be easier to understand if you put the simple case, i.e. Speed, first, followed by the more complicated one, i.e. Scale.
I am also wondering if it would make sense to add to the Speed chapter a sentence like:
"Note that the data rate on the transmission medium is proportional to the Speed factor, e.g. a content delivered with Speed=2 will use twice the data rate as the same content delivered with Speed=1."

And this brings me to another question: what about negative Speed values ? I didn't look up if they are generally prohibited somewhere, but since the text talks about negative scale values, it should at least mention negative speed values - if necessary stating that they are prohibited.

Helmut Bürklin


                        Helmut Bürklin
                        Senior technical advisor
                        Thomson R&D France
                        Technology group
                        1, Avenue Belle-Fontaine - CS17616
                        35576 Cesson-Sévigné, FRANCE
                        Tel : +33 (0) 2 99 27 36 27
                        Fax : +33 (0) 2 99 27 35 58
                        E-mail : [hidden email]



-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Magnus Westerlund
Sent: mardi 2 décembre 2008 13:46
To: mmusic (E-mail)
Subject: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

Below is an overview text about Speed and Scale. Trying to ensure that we all agree what Scale and Speed is supposed to do. So please review and see if you think you understand this. If it matches your understanding of the concepts and what need you see of either of the mechanisms. I want to ensure that we are on the same page and do see a need for both before updating the rest of the text regarding these subjects.

Media Delivery Manipulations

The basic playback functionality of RTSP is to request content for a particular range to be delivered to the client in a pace that enables playback as intended by the creator. However, RTSP can also manipulate how this delivery is done to the client in two ways.

   Scale: The ratio of media content delivered per unit playback time.

   Speed: The ratio of playback time delivered per unit of wall clock
          time.

So both affects the media delivery per time unit. However, they are manipulating two independent time scales and the effects are possible to combine.

Scale is used for fast forward or slow motion control as it changes the amount of content timescale that should be played back per time unit.
Scale > 1.0, means fast forward, e.g. Scale=2.0 results in that 2 seconds of content is played back every second of playback. Scale = 1.0 is the default value that is used if no Scale is specified, i.e.
playback at the contents original rate. Scale values between 0 and 1.0 is providing for slow motion. Scale can be negative to allow for reverse playback in either regular pace (Scale = -1.0) or fast backwards (Scale < -1.0) or slow motion backwards (-1.0 < Scale < 0).

In most cases the realization of scale means server side manipulation of the media to ensure that the client can actually play it back. These media manipulation and when they are needed are highly media type dependent. Lets exemplify with two common media types audio and video.

It is very difficult to modify the playback rate of audio. A maximum of 10-30% is possible by changing the pitch-rate of speech. Music goes out of tune if one tries to manipulate the playback rate by resampling it.
This is a well known problem and audio is commonly muted or played back in short segments with skips to keep up with the current playback point.

For video is possible to manipulate the number of frames that is displayed per second. But the rendering capabilities are often limited to certain frame rates. The decoding, handling capabilities and bitrate of received encoded content also limits the number of frames that can be delivered. Therefore when providing fast forward one generally picks a subset of the frames from the original content to be displayed. However, the video encoding methods use will commonly limit the possibilities on which frames that can be chosen and still be decoded by the receiver.

Due to the media restrictions a particular content will commonly be restricted to a limited set of possible scale ratios. To handle this correctly, RTSP has mechanism to indicate the supported Scale ratios for the content. To support aggregated or dynamic content where this may change during the ongoing session and dependent on the location within the content a mechanism for updating the media properties and the current used scale factor exist.

Speed affects how much of the playback timeline that is delivered in a given wall clock period. The default is Speed = 1 which is to deliver at the same rate the media is consumed. Speed > 1 means that the receiver will get content faster than it regularly would consume it. Speed < 1 means that it receives it slower than the regular consumption rate. This mechanism enables two general functionalities. Client side scale operations, i.e. the client receives all the frames and makes the adjustment to the playback locally. The second usage is to control its buffering of media. By specifying a speed over 1.0 the client can build up the amount of playback time it has present in its buffers to a level that is sufficient for its needs.
--

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Torbjörn Einarsson (KI/EAB)
Hi,

I agree that speed is conceptually simpler (and more easy to implement if bandwidth increase is allowed), so it may go first.

I think one should put in the work "time" in the definition of Scale, to make the ratio unitless.


   Speed: The ratio of playback time delivered per unit of wall clock time

   Scale: The ratio of media content time delivered per unit playback time


Furthermore, I think some examples referring to RTP timestamps would make thinks clearer. That text might be put at the end.

Here are some details for RTP, to illustrate how different time lines are influenced.

  For RTP, the playout time is always given by the RTP time stamps. The influence of Speed and Scale is

   Speed: RTP time stamps increase with average rate Speed*(RTP clock frequency).
          No change in relation between media playout speed and original media speed.
          The media is played at the original speed.

   Scale: RTP time stamps increase with average rate equal to RTP clock frequency.
          The relation between the RTP clock and the original media clock has been changed by a factor Scale.
          The media is played Scale times faster than the original speed.


Another thing is:
* What shall the server answer if it cannot do the scale asked for, but another close-by scale? For example, the client asks for scale 5, but the server can only support 3 and 6. Shall it respond with 6, or send an error message?

A final thing that needs a solution is the bandwidth handling in the case os Speed, or rather if we should have a trade-off between bandwidth and quality.

Shall it be allowed to send data faster than the bandwidth signalled in SDP (or alike)?
For early buffer filling, this is definitely useful. Furthermore, if the bandwidth limitations must be kept,
then the server must manipulate the streams, which may be very difficult and lead to undesirable results.
Maybe the client should be able to signal that it can accept adaptation? E.g.
* Speed 3, no-content adaptation
* Speed 3, keep band-width but allow adaptation
Increasing the bitrate above the session bandwidth is of course a potential congestion problem, so maybe we should simply state that the server should be careful to avoid congestiosn, e.g. by monitoring RTCP RR?

BR,
Torbjörn
--
Torbjörn Einarsson, MSc PhD
Mgr Media Protocols and Applications
Ericsson Research, Multimedia Technologies
Phone: +46 10 7172346
Mobile: +46 70 673 3424
[hidden email]

Ericsson AB
Färögatan 2-6
S-164 80 Stockholm, Sweden
www.ericsson.com
 
 
 



-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Burklin Helmut
Sent: den 2 december 2008 15:24
To: Magnus Westerlund; mmusic (E-mail)
Subject: Re: [MMUSIC] RTSP: Speed and Scale overview text

Hallo,
I think for the layman the text will be easier to understand if you put the simple case, i.e. Speed, first, followed by the more complicated one, i.e. Scale.
I am also wondering if it would make sense to add to the Speed chapter a sentence like:
"Note that the data rate on the transmission medium is proportional to the Speed factor, e.g. a content delivered with Speed=2 will use twice the data rate as the same content delivered with Speed=1."

And this brings me to another question: what about negative Speed values ? I didn't look up if they are generally prohibited somewhere, but since the text talks about negative scale values, it should at least mention negative speed values - if necessary stating that they are prohibited.

Helmut Bürklin


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Magnus Westerlund
Sent: mardi 2 décembre 2008 13:46
To: mmusic (E-mail)
Subject: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

Below is an overview text about Speed and Scale. Trying to ensure that we all agree what Scale and Speed is supposed to do. So please review and see if you think you understand this. If it matches your understanding of the concepts and what need you see of either of the mechanisms. I want to ensure that we are on the same page and do see a need for both before updating the rest of the text regarding these subjects.

Media Delivery Manipulations

The basic playback functionality of RTSP is to request content for a particular range to be delivered to the client in a pace that enables playback as intended by the creator. However, RTSP can also manipulate how this delivery is done to the client in two ways.

   Scale: The ratio of media content delivered per unit playback time.

   Speed: The ratio of playback time delivered per unit of wall clock
          time.

So both affects the media delivery per time unit. However, they are manipulating two independent time scales and the effects are possible to combine.

Scale is used for fast forward or slow motion control as it changes the amount of content timescale that should be played back per time unit.
Scale > 1.0, means fast forward, e.g. Scale=2.0 results in that 2 seconds of content is played back every second of playback. Scale = 1.0 is the default value that is used if no Scale is specified, i.e.
playback at the contents original rate. Scale values between 0 and 1.0 is providing for slow motion. Scale can be negative to allow for reverse playback in either regular pace (Scale = -1.0) or fast backwards (Scale < -1.0) or slow motion backwards (-1.0 < Scale < 0).

In most cases the realization of scale means server side manipulation of the media to ensure that the client can actually play it back. These media manipulation and when they are needed are highly media type dependent. Lets exemplify with two common media types audio and video.

It is very difficult to modify the playback rate of audio. A maximum of 10-30% is possible by changing the pitch-rate of speech. Music goes out of tune if one tries to manipulate the playback rate by resampling it.
This is a well known problem and audio is commonly muted or played back in short segments with skips to keep up with the current playback point.

For video is possible to manipulate the number of frames that is displayed per second. But the rendering capabilities are often limited to certain frame rates. The decoding, handling capabilities and bitrate of received encoded content also limits the number of frames that can be delivered. Therefore when providing fast forward one generally picks a subset of the frames from the original content to be displayed. However, the video encoding methods use will commonly limit the possibilities on which frames that can be chosen and still be decoded by the receiver.

Due to the media restrictions a particular content will commonly be restricted to a limited set of possible scale ratios. To handle this correctly, RTSP has mechanism to indicate the supported Scale ratios for the content. To support aggregated or dynamic content where this may change during the ongoing session and dependent on the location within the content a mechanism for updating the media properties and the current used scale factor exist.

Speed affects how much of the playback timeline that is delivered in a given wall clock period. The default is Speed = 1 which is to deliver at the same rate the media is consumed. Speed > 1 means that the receiver will get content faster than it regularly would consume it. Speed < 1 means that it receives it slower than the regular consumption rate. This mechanism enables two general functionalities. Client side scale operations, i.e. the client receives all the frames and makes the adjustment to the playback locally. The second usage is to control its buffering of media. By specifying a speed over 1.0 the client can build up the amount of playback time it has present in its buffers to a level that is sufficient for its needs.
--

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Burklin Helmut
In reply to this post by Magnus Westerlund
 Hm, Torbjörn,
I'm a little bit surprised. You say:
"   Speed: RTP time stamps increase with average rate Speed*(RTP clock frequency).
          No change in relation between media playout speed and original media speed.
          The media is played at the original speed."
I believed that we may use Speed for doing slow playout if, e.g., the server is not supporting the Scale. Or doing fast playout if the server doesn't support scale and the network supports the additional load.
I don't think that we shall impose here to the client at which rate he shall do the playout - this is under his whole responsability. Presentation time stamps indicate whent the content should be presented under nominal conditions - but here we are not under nominal conditions any more.
Note also that we have a special case for MPEG2-TS (RFC2250) where the timestamp is NOT a presentation time stamp.

For the other questions, I think they go beyond the introductory text Magnus had in mind. (though they are valuable: for the first one I think the server SHOULD choose the nearest value - whatever he considers to be nearest.

best regards
Helmut Bürklin

-----Original Message-----
From: Torbjörn Einarsson [mailto:[hidden email]]
Sent: mercredi 3 décembre 2008 09:46
To: mmusic (E-mail)
Cc: Burklin Helmut; Magnus Westerlund
Subject: RE: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

I agree that speed is conceptually simpler (and more easy to implement if bandwidth increase is allowed), so it may go first.

I think one should put in the work "time" in the definition of Scale, to make the ratio unitless.


   Speed: The ratio of playback time delivered per unit of wall clock time

   Scale: The ratio of media content time delivered per unit playback time


Furthermore, I think some examples referring to RTP timestamps would make thinks clearer. That text might be put at the end.

Here are some details for RTP, to illustrate how different time lines are influenced.

  For RTP, the playout time is always given by the RTP time stamps. The influence of Speed and Scale is

   Speed: RTP time stamps increase with average rate Speed*(RTP clock frequency).
          No change in relation between media playout speed and original media speed.
          The media is played at the original speed.

   Scale: RTP time stamps increase with average rate equal to RTP clock frequency.
          The relation between the RTP clock and the original media clock has been changed by a factor Scale.
          The media is played Scale times faster than the original speed.


Another thing is:
* What shall the server answer if it cannot do the scale asked for, but another close-by scale? For example, the client asks for scale 5, but the server can only support 3 and 6. Shall it respond with 6, or send an error message?

A final thing that needs a solution is the bandwidth handling in the case os Speed, or rather if we should have a trade-off between bandwidth and quality.

Shall it be allowed to send data faster than the bandwidth signalled in SDP (or alike)?
For early buffer filling, this is definitely useful. Furthermore, if the bandwidth limitations must be kept, then the server must manipulate the streams, which may be very difficult and lead to undesirable results.
Maybe the client should be able to signal that it can accept adaptation? E.g.
* Speed 3, no-content adaptation
* Speed 3, keep band-width but allow adaptation Increasing the bitrate above the session bandwidth is of course a potential congestion problem, so maybe we should simply state that the server should be careful to avoid congestiosn, e.g. by monitoring RTCP RR?

BR,
Torbjörn
--
Torbjörn Einarsson, MSc PhD
Mgr Media Protocols and Applications
Ericsson Research, Multimedia Technologies
Phone: +46 10 7172346
Mobile: +46 70 673 3424
[hidden email]

Ericsson AB
Färögatan 2-6
S-164 80 Stockholm, Sweden
www.ericsson.com
 
 
 



-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Burklin Helmut
Sent: den 2 december 2008 15:24
To: Magnus Westerlund; mmusic (E-mail)
Subject: Re: [MMUSIC] RTSP: Speed and Scale overview text

Hallo,
I think for the layman the text will be easier to understand if you put the simple case, i.e. Speed, first, followed by the more complicated one, i.e. Scale.
I am also wondering if it would make sense to add to the Speed chapter a sentence like:
"Note that the data rate on the transmission medium is proportional to the Speed factor, e.g. a content delivered with Speed=2 will use twice the data rate as the same content delivered with Speed=1."

And this brings me to another question: what about negative Speed values ? I didn't look up if they are generally prohibited somewhere, but since the text talks about negative scale values, it should at least mention negative speed values - if necessary stating that they are prohibited.

Helmut Bürklin


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Magnus Westerlund
Sent: mardi 2 décembre 2008 13:46
To: mmusic (E-mail)
Subject: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

Below is an overview text about Speed and Scale. Trying to ensure that we all agree what Scale and Speed is supposed to do. So please review and see if you think you understand this. If it matches your understanding of the concepts and what need you see of either of the mechanisms. I want to ensure that we are on the same page and do see a need for both before updating the rest of the text regarding these subjects.

Media Delivery Manipulations

The basic playback functionality of RTSP is to request content for a particular range to be delivered to the client in a pace that enables playback as intended by the creator. However, RTSP can also manipulate how this delivery is done to the client in two ways.

   Scale: The ratio of media content delivered per unit playback time.

   Speed: The ratio of playback time delivered per unit of wall clock
          time.

So both affects the media delivery per time unit. However, they are manipulating two independent time scales and the effects are possible to combine.

Scale is used for fast forward or slow motion control as it changes the amount of content timescale that should be played back per time unit.
Scale > 1.0, means fast forward, e.g. Scale=2.0 results in that 2 seconds of content is played back every second of playback. Scale = 1.0 is the default value that is used if no Scale is specified, i.e.
playback at the contents original rate. Scale values between 0 and 1.0 is providing for slow motion. Scale can be negative to allow for reverse playback in either regular pace (Scale = -1.0) or fast backwards (Scale < -1.0) or slow motion backwards (-1.0 < Scale < 0).

In most cases the realization of scale means server side manipulation of the media to ensure that the client can actually play it back. These media manipulation and when they are needed are highly media type dependent. Lets exemplify with two common media types audio and video.

It is very difficult to modify the playback rate of audio. A maximum of 10-30% is possible by changing the pitch-rate of speech. Music goes out of tune if one tries to manipulate the playback rate by resampling it.
This is a well known problem and audio is commonly muted or played back in short segments with skips to keep up with the current playback point.

For video is possible to manipulate the number of frames that is displayed per second. But the rendering capabilities are often limited to certain frame rates. The decoding, handling capabilities and bitrate of received encoded content also limits the number of frames that can be delivered. Therefore when providing fast forward one generally picks a subset of the frames from the original content to be displayed. However, the video encoding methods use will commonly limit the possibilities on which frames that can be chosen and still be decoded by the receiver.

Due to the media restrictions a particular content will commonly be restricted to a limited set of possible scale ratios. To handle this correctly, RTSP has mechanism to indicate the supported Scale ratios for the content. To support aggregated or dynamic content where this may change during the ongoing session and dependent on the location within the content a mechanism for updating the media properties and the current used scale factor exist.

Speed affects how much of the playback timeline that is delivered in a given wall clock period. The default is Speed = 1 which is to deliver at the same rate the media is consumed. Speed > 1 means that the receiver will get content faster than it regularly would consume it. Speed < 1 means that it receives it slower than the regular consumption rate. This mechanism enables two general functionalities. Client side scale operations, i.e. the client receives all the frames and makes the adjustment to the playback locally. The second usage is to control its buffering of media. By specifying a speed over 1.0 the client can build up the amount of playback time it has present in its buffers to a level that is sufficient for its needs.
--

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Torbjörn Einarsson (KI/EAB)
You're of course right. We cannot specify the client behaviour. It should be free to do adaptive playout or something similar. The main purpose of the suggested text was to make it clearer what is changed in the different cases. Do you think we should put in any text on RTP at all?

BR,
Torbjörn  

-----Original Message-----
From: Burklin Helmut [mailto:[hidden email]]
Sent: den 3 december 2008 10:09
To: Torbjörn Einarsson; mmusic (E-mail)
Cc: Magnus Westerlund
Subject: RE: [MMUSIC] RTSP: Speed and Scale overview text

 Hm, Torbjörn,
I'm a little bit surprised. You say:
"   Speed: RTP time stamps increase with average rate Speed*(RTP clock frequency).
          No change in relation between media playout speed and original media speed.
          The media is played at the original speed."
I believed that we may use Speed for doing slow playout if, e.g., the server is not supporting the Scale. Or doing fast playout if the server doesn't support scale and the network supports the additional load.
I don't think that we shall impose here to the client at which rate he shall do the playout - this is under his whole responsability. Presentation time stamps indicate whent the content should be presented under nominal conditions - but here we are not under nominal conditions any more.
Note also that we have a special case for MPEG2-TS (RFC2250) where the timestamp is NOT a presentation time stamp.

For the other questions, I think they go beyond the introductory text Magnus had in mind. (though they are valuable: for the first one I think the server SHOULD choose the nearest value - whatever he considers to be nearest.

best regards
Helmut Bürklin

-----Original Message-----
From: Torbjörn Einarsson [mailto:[hidden email]]
Sent: mercredi 3 décembre 2008 09:46
To: mmusic (E-mail)
Cc: Burklin Helmut; Magnus Westerlund
Subject: RE: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

I agree that speed is conceptually simpler (and more easy to implement if bandwidth increase is allowed), so it may go first.

I think one should put in the work "time" in the definition of Scale, to make the ratio unitless.


   Speed: The ratio of playback time delivered per unit of wall clock time

   Scale: The ratio of media content time delivered per unit playback time


Furthermore, I think some examples referring to RTP timestamps would make thinks clearer. That text might be put at the end.

Here are some details for RTP, to illustrate how different time lines are influenced.

  For RTP, the playout time is always given by the RTP time stamps. The influence of Speed and Scale is

   Speed: RTP time stamps increase with average rate Speed*(RTP clock frequency).
          No change in relation between media playout speed and original media speed.
          The media is played at the original speed.

   Scale: RTP time stamps increase with average rate equal to RTP clock frequency.
          The relation between the RTP clock and the original media clock has been changed by a factor Scale.
          The media is played Scale times faster than the original speed.


Another thing is:
* What shall the server answer if it cannot do the scale asked for, but another close-by scale? For example, the client asks for scale 5, but the server can only support 3 and 6. Shall it respond with 6, or send an error message?

A final thing that needs a solution is the bandwidth handling in the case os Speed, or rather if we should have a trade-off between bandwidth and quality.

Shall it be allowed to send data faster than the bandwidth signalled in SDP (or alike)?
For early buffer filling, this is definitely useful. Furthermore, if the bandwidth limitations must be kept, then the server must manipulate the streams, which may be very difficult and lead to undesirable results.
Maybe the client should be able to signal that it can accept adaptation? E.g.
* Speed 3, no-content adaptation
* Speed 3, keep band-width but allow adaptation Increasing the bitrate above the session bandwidth is of course a potential congestion problem, so maybe we should simply state that the server should be careful to avoid congestiosn, e.g. by monitoring RTCP RR?

BR,
Torbjörn
--
Torbjörn Einarsson, MSc PhD
Mgr Media Protocols and Applications
Ericsson Research, Multimedia Technologies
Phone: +46 10 7172346
Mobile: +46 70 673 3424
[hidden email]

Ericsson AB
Färögatan 2-6
S-164 80 Stockholm, Sweden
www.ericsson.com
 
 
 



-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Burklin Helmut
Sent: den 2 december 2008 15:24
To: Magnus Westerlund; mmusic (E-mail)
Subject: Re: [MMUSIC] RTSP: Speed and Scale overview text

Hallo,
I think for the layman the text will be easier to understand if you put the simple case, i.e. Speed, first, followed by the more complicated one, i.e. Scale.
I am also wondering if it would make sense to add to the Speed chapter a sentence like:
"Note that the data rate on the transmission medium is proportional to the Speed factor, e.g. a content delivered with Speed=2 will use twice the data rate as the same content delivered with Speed=1."

And this brings me to another question: what about negative Speed values ? I didn't look up if they are generally prohibited somewhere, but since the text talks about negative scale values, it should at least mention negative speed values - if necessary stating that they are prohibited.

Helmut Bürklin


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Magnus Westerlund
Sent: mardi 2 décembre 2008 13:46
To: mmusic (E-mail)
Subject: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

Below is an overview text about Speed and Scale. Trying to ensure that we all agree what Scale and Speed is supposed to do. So please review and see if you think you understand this. If it matches your understanding of the concepts and what need you see of either of the mechanisms. I want to ensure that we are on the same page and do see a need for both before updating the rest of the text regarding these subjects.

Media Delivery Manipulations

The basic playback functionality of RTSP is to request content for a particular range to be delivered to the client in a pace that enables playback as intended by the creator. However, RTSP can also manipulate how this delivery is done to the client in two ways.

   Scale: The ratio of media content delivered per unit playback time.

   Speed: The ratio of playback time delivered per unit of wall clock
          time.

So both affects the media delivery per time unit. However, they are manipulating two independent time scales and the effects are possible to combine.

Scale is used for fast forward or slow motion control as it changes the amount of content timescale that should be played back per time unit.
Scale > 1.0, means fast forward, e.g. Scale=2.0 results in that 2 seconds of content is played back every second of playback. Scale = 1.0 is the default value that is used if no Scale is specified, i.e.
playback at the contents original rate. Scale values between 0 and 1.0 is providing for slow motion. Scale can be negative to allow for reverse playback in either regular pace (Scale = -1.0) or fast backwards (Scale < -1.0) or slow motion backwards (-1.0 < Scale < 0).

In most cases the realization of scale means server side manipulation of the media to ensure that the client can actually play it back. These media manipulation and when they are needed are highly media type dependent. Lets exemplify with two common media types audio and video.

It is very difficult to modify the playback rate of audio. A maximum of 10-30% is possible by changing the pitch-rate of speech. Music goes out of tune if one tries to manipulate the playback rate by resampling it.
This is a well known problem and audio is commonly muted or played back in short segments with skips to keep up with the current playback point.

For video is possible to manipulate the number of frames that is displayed per second. But the rendering capabilities are often limited to certain frame rates. The decoding, handling capabilities and bitrate of received encoded content also limits the number of frames that can be delivered. Therefore when providing fast forward one generally picks a subset of the frames from the original content to be displayed. However, the video encoding methods use will commonly limit the possibilities on which frames that can be chosen and still be decoded by the receiver.

Due to the media restrictions a particular content will commonly be restricted to a limited set of possible scale ratios. To handle this correctly, RTSP has mechanism to indicate the supported Scale ratios for the content. To support aggregated or dynamic content where this may change during the ongoing session and dependent on the location within the content a mechanism for updating the media properties and the current used scale factor exist.

Speed affects how much of the playback timeline that is delivered in a given wall clock period. The default is Speed = 1 which is to deliver at the same rate the media is consumed. Speed > 1 means that the receiver will get content faster than it regularly would consume it. Speed < 1 means that it receives it slower than the regular consumption rate. This mechanism enables two general functionalities. Client side scale operations, i.e. the client receives all the frames and makes the adjustment to the playback locally. The second usage is to control its buffering of media. By specifying a speed over 1.0 the client can build up the amount of playback time it has present in its buffers to a level that is sufficient for its needs.
--

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Burklin Helmut
In reply to this post by Magnus Westerlund
 "Do you think we should put in any text on RTP at all?"
I don't know, since I didn't look what is the precise behaviour with RFC2250.
BR
Helmut

-----Original Message-----
From: Torbjörn Einarsson [mailto:[hidden email]]
Sent: mercredi 3 décembre 2008 10:19
To: Burklin Helmut; mmusic (E-mail)
Cc: Magnus Westerlund
Subject: RE: [MMUSIC] RTSP: Speed and Scale overview text

You're of course right. We cannot specify the client behaviour. It should be free to do adaptive playout or something similar. The main purpose of the suggested text was to make it clearer what is changed in the different cases. Do you think we should put in any text on RTP at all?

BR,
Torbjörn  

-----Original Message-----
From: Burklin Helmut [mailto:[hidden email]]
Sent: den 3 december 2008 10:09
To: Torbjörn Einarsson; mmusic (E-mail)
Cc: Magnus Westerlund
Subject: RE: [MMUSIC] RTSP: Speed and Scale overview text

 Hm, Torbjörn,
I'm a little bit surprised. You say:
"   Speed: RTP time stamps increase with average rate Speed*(RTP clock frequency).
          No change in relation between media playout speed and original media speed.
          The media is played at the original speed."
I believed that we may use Speed for doing slow playout if, e.g., the server is not supporting the Scale. Or doing fast playout if the server doesn't support scale and the network supports the additional load.
I don't think that we shall impose here to the client at which rate he shall do the playout - this is under his whole responsability. Presentation time stamps indicate whent the content should be presented under nominal conditions - but here we are not under nominal conditions any more.
Note also that we have a special case for MPEG2-TS (RFC2250) where the timestamp is NOT a presentation time stamp.

For the other questions, I think they go beyond the introductory text Magnus had in mind. (though they are valuable: for the first one I think the server SHOULD choose the nearest value - whatever he considers to be nearest.

best regards
Helmut Bürklin

-----Original Message-----
From: Torbjörn Einarsson [mailto:[hidden email]]
Sent: mercredi 3 décembre 2008 09:46
To: mmusic (E-mail)
Cc: Burklin Helmut; Magnus Westerlund
Subject: RE: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

I agree that speed is conceptually simpler (and more easy to implement if bandwidth increase is allowed), so it may go first.

I think one should put in the work "time" in the definition of Scale, to make the ratio unitless.


   Speed: The ratio of playback time delivered per unit of wall clock time

   Scale: The ratio of media content time delivered per unit playback time


Furthermore, I think some examples referring to RTP timestamps would make thinks clearer. That text might be put at the end.

Here are some details for RTP, to illustrate how different time lines are influenced.

  For RTP, the playout time is always given by the RTP time stamps. The influence of Speed and Scale is

   Speed: RTP time stamps increase with average rate Speed*(RTP clock frequency).
          No change in relation between media playout speed and original media speed.
          The media is played at the original speed.

   Scale: RTP time stamps increase with average rate equal to RTP clock frequency.
          The relation between the RTP clock and the original media clock has been changed by a factor Scale.
          The media is played Scale times faster than the original speed.


Another thing is:
* What shall the server answer if it cannot do the scale asked for, but another close-by scale? For example, the client asks for scale 5, but the server can only support 3 and 6. Shall it respond with 6, or send an error message?

A final thing that needs a solution is the bandwidth handling in the case os Speed, or rather if we should have a trade-off between bandwidth and quality.

Shall it be allowed to send data faster than the bandwidth signalled in SDP (or alike)?
For early buffer filling, this is definitely useful. Furthermore, if the bandwidth limitations must be kept, then the server must manipulate the streams, which may be very difficult and lead to undesirable results.
Maybe the client should be able to signal that it can accept adaptation? E.g.
* Speed 3, no-content adaptation
* Speed 3, keep band-width but allow adaptation Increasing the bitrate above the session bandwidth is of course a potential congestion problem, so maybe we should simply state that the server should be careful to avoid congestiosn, e.g. by monitoring RTCP RR?

BR,
Torbjörn
--
Torbjörn Einarsson, MSc PhD
Mgr Media Protocols and Applications
Ericsson Research, Multimedia Technologies
Phone: +46 10 7172346
Mobile: +46 70 673 3424
[hidden email]

Ericsson AB
Färögatan 2-6
S-164 80 Stockholm, Sweden
www.ericsson.com
 
 
 



-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Burklin Helmut
Sent: den 2 december 2008 15:24
To: Magnus Westerlund; mmusic (E-mail)
Subject: Re: [MMUSIC] RTSP: Speed and Scale overview text

Hallo,
I think for the layman the text will be easier to understand if you put the simple case, i.e. Speed, first, followed by the more complicated one, i.e. Scale.
I am also wondering if it would make sense to add to the Speed chapter a sentence like:
"Note that the data rate on the transmission medium is proportional to the Speed factor, e.g. a content delivered with Speed=2 will use twice the data rate as the same content delivered with Speed=1."

And this brings me to another question: what about negative Speed values ? I didn't look up if they are generally prohibited somewhere, but since the text talks about negative scale values, it should at least mention negative speed values - if necessary stating that they are prohibited.

Helmut Bürklin


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Magnus Westerlund
Sent: mardi 2 décembre 2008 13:46
To: mmusic (E-mail)
Subject: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

Below is an overview text about Speed and Scale. Trying to ensure that we all agree what Scale and Speed is supposed to do. So please review and see if you think you understand this. If it matches your understanding of the concepts and what need you see of either of the mechanisms. I want to ensure that we are on the same page and do see a need for both before updating the rest of the text regarding these subjects.

Media Delivery Manipulations

The basic playback functionality of RTSP is to request content for a particular range to be delivered to the client in a pace that enables playback as intended by the creator. However, RTSP can also manipulate how this delivery is done to the client in two ways.

   Scale: The ratio of media content delivered per unit playback time.

   Speed: The ratio of playback time delivered per unit of wall clock
          time.

So both affects the media delivery per time unit. However, they are manipulating two independent time scales and the effects are possible to combine.

Scale is used for fast forward or slow motion control as it changes the amount of content timescale that should be played back per time unit.
Scale > 1.0, means fast forward, e.g. Scale=2.0 results in that 2 seconds of content is played back every second of playback. Scale = 1.0 is the default value that is used if no Scale is specified, i.e.
playback at the contents original rate. Scale values between 0 and 1.0 is providing for slow motion. Scale can be negative to allow for reverse playback in either regular pace (Scale = -1.0) or fast backwards (Scale < -1.0) or slow motion backwards (-1.0 < Scale < 0).

In most cases the realization of scale means server side manipulation of the media to ensure that the client can actually play it back. These media manipulation and when they are needed are highly media type dependent. Lets exemplify with two common media types audio and video.

It is very difficult to modify the playback rate of audio. A maximum of 10-30% is possible by changing the pitch-rate of speech. Music goes out of tune if one tries to manipulate the playback rate by resampling it.
This is a well known problem and audio is commonly muted or played back in short segments with skips to keep up with the current playback point.

For video is possible to manipulate the number of frames that is displayed per second. But the rendering capabilities are often limited to certain frame rates. The decoding, handling capabilities and bitrate of received encoded content also limits the number of frames that can be delivered. Therefore when providing fast forward one generally picks a subset of the frames from the original content to be displayed. However, the video encoding methods use will commonly limit the possibilities on which frames that can be chosen and still be decoded by the receiver.

Due to the media restrictions a particular content will commonly be restricted to a limited set of possible scale ratios. To handle this correctly, RTSP has mechanism to indicate the supported Scale ratios for the content. To support aggregated or dynamic content where this may change during the ongoing session and dependent on the location within the content a mechanism for updating the media properties and the current used scale factor exist.

Speed affects how much of the playback timeline that is delivered in a given wall clock period. The default is Speed = 1 which is to deliver at the same rate the media is consumed. Speed > 1 means that the receiver will get content faster than it regularly would consume it. Speed < 1 means that it receives it slower than the regular consumption rate. This mechanism enables two general functionalities. Client side scale operations, i.e. the client receives all the frames and makes the adjustment to the playback locally. The second usage is to control its buffering of media. By specifying a speed over 1.0 the client can build up the amount of playback time it has present in its buffers to a level that is sufficient for its needs.
--

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Imed.Bouazizi
In reply to this post by Burklin Helmut
RE: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

in order not to overload the usage of Speed, I would propose to define it in relationship to bandwidth only and not time.
This would make it clear that no content adaptation is to be performed by the server in reaction to speed changes and that only transmission scheduling is affected.

By decoupling both concepts, client and server are able to negotiate content adaptation in a more explicit manner. Handling of Speed requests that result in congestion is also made easier by adjusting output bitrate to channel bandwidth and then determining the closest feasible Speed.

Desirable quality adaptation may still be achieved explicitly by switching to a lower/higher bitrate representation of the same content or otherwise.

Input                     Scaled Content                Modified Speed
Bitrate +------------+       Bitrate    +--------------+   Bitrate   +------------+
------->|   Scaler   |----------------->|  Scheduler   |------------>|  Channel   |
        +------------+                  +--------------+             +------------+
Modified Speed = min(Speed,available bandwidth/Scaled Content Bitrate)
Modified Speed Bitrate = Modified Speed * Scaled Content Bitrate 


Regards,
Imed Bouazizi

-----Original Message-----
From: [hidden email] on behalf of ext Burklin Helmut
Sent: Wed 12/3/2008 11:09 AM
To: Torbjörn Einarsson; mmusic (E-mail)
Cc: Magnus Westerlund
Subject: Re: [MMUSIC] RTSP: Speed and Scale overview text


 Hm, Torbjörn,
I'm a little bit surprised. You say:
"   Speed: RTP time stamps increase with average rate Speed*(RTP clock frequency).
          No change in relation between media playout speed and original media speed.
          The media is played at the original speed."
I believed that we may use Speed for doing slow playout if, e.g., the server is not supporting the Scale. Or doing fast playout if the server doesn't support scale and the network supports the additional load.
I don't think that we shall impose here to the client at which rate he shall do the playout - this is under his whole responsability. Presentation time stamps indicate whent the content should be presented under nominal conditions - but here we are not under nominal conditions any more.
Note also that we have a special case for MPEG2-TS (RFC2250) where the timestamp is NOT a presentation time stamp.

For the other questions, I think they go beyond the introductory text Magnus had in mind. (though they are valuable: for the first one I think the server SHOULD choose the nearest value - whatever he considers to be nearest.

best regards
Helmut Bürklin

-----Original Message-----
From: Torbjörn Einarsson [[hidden email]]
Sent: mercredi 3 décembre 2008 09:46
To: mmusic (E-mail)
Cc: Burklin Helmut; Magnus Westerlund
Subject: RE: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

I agree that speed is conceptually simpler (and more easy to implement if bandwidth increase is allowed), so it may go first.

I think one should put in the work "time" in the definition of Scale, to make the ratio unitless.


   Speed: The ratio of playback time delivered per unit of wall clock time

   Scale: The ratio of media content time delivered per unit playback time


Furthermore, I think some examples referring to RTP timestamps would make thinks clearer. That text might be put at the end.

Here are some details for RTP, to illustrate how different time lines are influenced.

  For RTP, the playout time is always given by the RTP time stamps. The influence of Speed and Scale is

   Speed: RTP time stamps increase with average rate Speed*(RTP clock frequency).
          No change in relation between media playout speed and original media speed.
          The media is played at the original speed.

   Scale: RTP time stamps increase with average rate equal to RTP clock frequency.
          The relation between the RTP clock and the original media clock has been changed by a factor Scale.
          The media is played Scale times faster than the original speed.


Another thing is:
* What shall the server answer if it cannot do the scale asked for, but another close-by scale? For example, the client asks for scale 5, but the server can only support 3 and 6. Shall it respond with 6, or send an error message?

A final thing that needs a solution is the bandwidth handling in the case os Speed, or rather if we should have a trade-off between bandwidth and quality.

Shall it be allowed to send data faster than the bandwidth signalled in SDP (or alike)?
For early buffer filling, this is definitely useful. Furthermore, if the bandwidth limitations must be kept, then the server must manipulate the streams, which may be very difficult and lead to undesirable results.
Maybe the client should be able to signal that it can accept adaptation? E.g.
* Speed 3, no-content adaptation
* Speed 3, keep band-width but allow adaptation Increasing the bitrate above the session bandwidth is of course a potential congestion problem, so maybe we should simply state that the server should be careful to avoid congestiosn, e.g. by monitoring RTCP RR?

BR,
Torbjörn
--
Torbjörn Einarsson, MSc PhD
Mgr Media Protocols and Applications
Ericsson Research, Multimedia Technologies
Phone: +46 10 7172346
Mobile: +46 70 673 3424
[hidden email]

Ericsson AB
Färögatan 2-6
S-164 80 Stockholm, Sweden
www.ericsson.com






-----Original Message-----
From: [hidden email] [[hidden email]] On Behalf Of Burklin Helmut
Sent: den 2 december 2008 15:24
To: Magnus Westerlund; mmusic (E-mail)
Subject: Re: [MMUSIC] RTSP: Speed and Scale overview text

Hallo,
I think for the layman the text will be easier to understand if you put the simple case, i.e. Speed, first, followed by the more complicated one, i.e. Scale.
I am also wondering if it would make sense to add to the Speed chapter a sentence like:
"Note that the data rate on the transmission medium is proportional to the Speed factor, e.g. a content delivered with Speed=2 will use twice the data rate as the same content delivered with Speed=1."

And this brings me to another question: what about negative Speed values ? I didn't look up if they are generally prohibited somewhere, but since the text talks about negative scale values, it should at least mention negative speed values - if necessary stating that they are prohibited.

Helmut Bürklin


-----Original Message-----
From: [hidden email] [[hidden email]] On Behalf Of Magnus Westerlund
Sent: mardi 2 décembre 2008 13:46
To: mmusic (E-mail)
Subject: [MMUSIC] RTSP: Speed and Scale overview text

Hi,

Below is an overview text about Speed and Scale. Trying to ensure that we all agree what Scale and Speed is supposed to do. So please review and see if you think you understand this. If it matches your understanding of the concepts and what need you see of either of the mechanisms. I want to ensure that we are on the same page and do see a need for both before updating the rest of the text regarding these subjects.

Media Delivery Manipulations

The basic playback functionality of RTSP is to request content for a particular range to be delivered to the client in a pace that enables playback as intended by the creator. However, RTSP can also manipulate how this delivery is done to the client in two ways.

   Scale: The ratio of media content delivered per unit playback time.

   Speed: The ratio of playback time delivered per unit of wall clock
          time.

So both affects the media delivery per time unit. However, they are manipulating two independent time scales and the effects are possible to combine.

Scale is used for fast forward or slow motion control as it changes the amount of content timescale that should be played back per time unit.
Scale > 1.0, means fast forward, e.g. Scale=2.0 results in that 2 seconds of content is played back every second of playback. Scale = 1.0 is the default value that is used if no Scale is specified, i.e.
playback at the contents original rate. Scale values between 0 and 1.0 is providing for slow motion. Scale can be negative to allow for reverse playback in either regular pace (Scale = -1.0) or fast backwards (Scale < -1.0) or slow motion backwards (-1.0 < Scale < 0).

In most cases the realization of scale means server side manipulation of the media to ensure that the client can actually play it back. These media manipulation and when they are needed are highly media type dependent. Lets exemplify with two common media types audio and video.

It is very difficult to modify the playback rate of audio. A maximum of 10-30% is possible by changing the pitch-rate of speech. Music goes out of tune if one tries to manipulate the playback rate by resampling it.
This is a well known problem and audio is commonly muted or played back in short segments with skips to keep up with the current playback point.

For video is possible to manipulate the number of frames that is displayed per second. But the rendering capabilities are often limited to certain frame rates. The decoding, handling capabilities and bitrate of received encoded content also limits the number of frames that can be delivered. Therefore when providing fast forward one generally picks a subset of the frames from the original content to be displayed. However, the video encoding methods use will commonly limit the possibilities on which frames that can be chosen and still be decoded by the receiver.

Due to the media restrictions a particular content will commonly be restricted to a limited set of possible scale ratios. To handle this correctly, RTSP has mechanism to indicate the supported Scale ratios for the content. To support aggregated or dynamic content where this may change during the ongoing session and dependent on the location within the content a mechanism for updating the media properties and the current used scale factor exist.

Speed affects how much of the playback timeline that is delivered in a given wall clock period. The default is Speed = 1 which is to deliver at the same rate the media is consumed. Speed > 1 means that the receiver will get content faster than it regularly would consume it. Speed < 1 means that it receives it slower than the regular consumption rate. This mechanism enables two general functionalities. Client side scale operations, i.e. the client receives all the frames and makes the adjustment to the playback locally. The second usage is to control its buffering of media. By specifying a speed over 1.0 the client can build up the amount of playback time it has present in its buffers to a level that is sufficient for its needs.
--

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic





_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Magnus Westerlund
In reply to this post by Torbjörn Einarsson (KI/EAB)
Torbjörn Einarsson skrev:
> You're of course right. We cannot specify the client behaviour. It should be free to do adaptive playout or something similar. The main purpose of the suggested text was to make it clearer what is changed in the different cases. Do you think we should put in any text on RTP at all?

I think there will be text on this, but not in this section. The right
spot to put RTP related discussion on Scale and Speed are in the
Appendix where media delivery using RTP is defined.

Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Magnus Westerlund
In reply to this post by Burklin Helmut
Burklin Helmut skrev:
> Hallo,
> I think for the layman the text will be easier to understand if you put the simple case, i.e. Speed, first, followed by the more complicated one, i.e. Scale.

I don't quite understand why this would be simpler. One of the reasons
for doing Speed is to do client side scale operations. Without
explaining Scale first I can't simple reference that as a reason. So
putting speed first doesn't make the text easier to follow in my view.
At least not without a major rewrite.

> I am also wondering if it would make sense to add to the Speed chapter a sentence like:
> "Note that the data rate on the transmission medium is proportional to the Speed factor, e.g. a content delivered with Speed=2 will use twice the data rate as the same content delivered with Speed=1."

That is what I am coming at in the end. Yes, that will be true for a
naive implementation. But for this to work in reality this can't be
universally true. It can only be true in cases where the network path
between server and client can support this. Otherwise the server will
have to adopt the media to fit within the available resources.

Thus as that is not true I don't want to add such a sentence.

>
> And this brings me to another question: what about negative Speed values ? I didn't look up if they are generally prohibited somewhere, but since the text talks about negative scale values, it should at least mention negative speed values - if necessary stating that they are prohibited.
>

Negative speed value has no meaning. Just as Scale=0 is equal to pause
and that shouldn't be allowed either as using Pause makes more sense. I
have added a sentence making this clear.

Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Magnus Westerlund
In reply to this post by Imed.Bouazizi
[hidden email] skrev:

> Hi,
>
> in order not to overload the usage of Speed, I would propose to define
> it in relationship to bandwidth only and not time.
> This would make it clear that no content adaptation is to be performed
> by the server in reaction to speed changes and that only transmission
> scheduling is affected.
>
> By decoupling both concepts, client and server are able to negotiate
> content adaptation in a more explicit manner. Handling of Speed requests
> that result in congestion is also made easier by adjusting output
> bitrate to channel bandwidth and then determining the closest feasible
> Speed.
>
> Desirable quality adaptation may still be achieved explicitly by
> switching to a lower/higher bitrate representation of the same content
> or otherwise.
>
> Input                     Scaled Content                Modified Speed
> Bitrate +------------+       Bitrate    +--------------+   Bitrate  
> +------------+
> ------->|   Scaler   |----------------->|  Scheduler   |------------>|
> Channel   |
>         +------------+                  +--------------+            
> +------------+
> Modified Speed = min(Speed,available bandwidth/Scaled Content Bitrate)
> Modified Speed Bitrate = Modified Speed * Scaled Content Bitrate
>

Imed,

I think the current definition of Speed does managed to have a
definition that isn't overloaded at all. Speed affects the amount of
nominal playback time that is delivered per time. Yes, reducing speed
will have the affect on bandwidth restricted paths to enable higher
quality. But the definition is clear.

Considering your definition of modified speed, that will be a dynamic
value that changes during the delivery as the value for available BW
fluctuates.

I think we are getting to the core of the issues around speed. Namely
what it is to be used for. Opportunistic buffer filling, in which case
your proposal makes total sense, or if one actually like to make a local
scaling operation where strict following the given speed value is more
important.

My personal view is that the client buffer management should be solved
with some solution that is more dynamic than RTSP.

Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Burklin Helmut
In reply to this post by Magnus Westerlund
 Magnus,
I apologize to have jumped in a bit hurriedly, I probably have
missed some prior agreements. Nevertheless let me ask a
question on your answer:

>
> > I am also wondering if it would make sense to add to the
> > Speed chapter a sentence like:
> > "Note that the data rate on the transmission medium is
> > proportional to the Speed factor, e.g. a content delivered
> > with Speed=2 will use twice the data rate as the same content
> > delivered with Speed=1."
>
> That is what I am coming at in the end. Yes, that will be
> true for a naive implementation. But for this to work in
> reality this can't be universally true. It can only be true
> in cases where the network path between server and client can
> support this. Otherwise the server will have to adopt the
> media to fit within the available resources.
>
> Thus as that is not true I don't want to add such a sentence.
>

So, in case the network path doesn't support the load, what will happen?
You say the server will "adapt the media" (typo corrected) - isn't this
reserved to the Scale case only ?
So do you think that the server that declares not to be able to scale,
but only to speed, shall scale anyway ?
You guess that I don't think this is a good idea (but as I said, I came
in late to this discussion)

Thank you for clarification

Helmut Bürklin

                        Helmut Bürklin
                        Senior technical advisor
                        Thomson R&D France
                        Technology group
                        1, Avenue Belle-Fontaine - CS17616
                        35576 Cesson-Sévigné, FRANCE
                        Tel : +33 (0) 2 99 27 36 27
                        Fax : +33 (0) 2 99 27 35 58
                        E-mail : [hidden email]


_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Magnus Westerlund
Burklin Helmut skrev:

>  Magnus,
> I apologize to have jumped in a bit hurriedly, I probably have
> missed some prior agreements. Nevertheless let me ask a
> question on your answer:
>
>>> I am also wondering if it would make sense to add to the
>>> Speed chapter a sentence like:
>>> "Note that the data rate on the transmission medium is
>>> proportional to the Speed factor, e.g. a content delivered
>>> with Speed=2 will use twice the data rate as the same content
>>> delivered with Speed=1."
>> That is what I am coming at in the end. Yes, that will be
>> true for a naive implementation. But for this to work in
>> reality this can't be universally true. It can only be true
>> in cases where the network path between server and client can
>> support this. Otherwise the server will have to adopt the
>> media to fit within the available resources.
>>
>> Thus as that is not true I don't want to add such a sentence.
>>
>
> So, in case the network path doesn't support the load, what will happen?
> You say the server will "adapt the media" (typo corrected) - isn't this
> reserved to the Scale case only ?

No, that is one the cases I want to stress here. Speed affects the
amount of playback time that is delivered per wall clock time. In a
naive implementation using a speed value that is larger than one will
increase the bandwidth. However, the point I am making is that you can't
do a naive implementation of this because it affects how your fulfill
the congestion control requirement.

When streaming media over the Internet you do need to be a good citizen
and thus adopt the media stream to the available resource. This is
independent if Speed is used or not. So when speed is used on has to
take into consideration that the server will attempt to increase the
bandwidth in relation to the speed value. However, if that isn't
available the server needs to keep the bandwidth with the available
resources. As I see it it can do this two ways:

1. Reduce the bits per playback time, i.e. reduce the quality of the
media encoding, without changing the actual speed ratio.

2. Reduce the speed ratio down to what is supportable as Imed proposed.

So one comes to the question of what should be immutable in this case.
For 1 it is the selected speed value. I see that appropriate if one does
local scale operations. In 2 one tries to make the quality immutable
which is more suitable if one does opportunistic buffer filling.
However, the quality isn't immutable, because if the network path can't
support Speed=1.0 either then the quality have to go down.

So the question remains, what usage are there for speed out there?

> So do you think that the server that declares not to be able to scale,
> but only to speed, shall scale anyway ?

No, but be able to do bit-rate adaptation. The most common solution to
this is after all bit-stream switching. The server have several version
of the encoding of the same content and is able to switch between the
bit-rates at regular points in the media stream. There are more advanced
solutions but they usually requires on the fly transcoding which is
quite resource demanding.

> You guess that I don't think this is a good idea (but as I said, I came
> in late to this discussion)
>

I will not comment on this until we are certain which the problem is,
scaling or bit-rate adaptation.

Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Rémi Denis-Courmont-7
In reply to this post by Imed.Bouazizi
On Wednesday 03 December 2008 12:32:35 ext [hidden email], you wrote:

> Hi,
>
> in order not to overload the usage of Speed, I would propose to define it
> in relationship to bandwidth only and not time. This would make it clear
> that no content adaptation is to be performed by the server in reaction to
> speed changes and that only transmission scheduling is affected.
>
> By decoupling both concepts, client and server are able to negotiate
> content adaptation in a more explicit manner. Handling of Speed requests
> that result in congestion is also made easier by adjusting output bitrate
> to channel bandwidth and then determining the closest feasible Speed.

Bandwidth is fundamentally tied to time, by its very definition. Decoupling
them makes no sense to me, and the very idea is utmostly confusing to me.

--
Rémi Denis-Courmont
Maemo Software, Nokia Devices R&D
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Magnus Westerlund
In reply to this post by Magnus Westerlund
Hi,

This is an email to trying to attempt to summarize the situation.

I think people are agreeing on the scale definition and that it is a
useful tool in some situations. I will therefore draft a spec solution
that tries to fix the known issues, including a media properties
attribute that carries a list of supported scale values for the media.

When it comes to Speed although people seems to agree on a very high
level on what speed is there is still no conclusion on why it is needed,
if at all. We need more input into why Speed is used and for what.
Without that understanding make clear specification text becomes impossible.

Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic
Reply | Threaded
Open this post in threaded view
|

Re: RTSP: Speed and Scale overview text

Magnus Westerlund
Hi,

I have received some input and have made an attempt for specification
text that covers several of the use cases. Here is the text that
currently is proposed:

Excerpt from Section 2.4.1:

   Speed affects how much of the playback timeline that is delivered in
   a given wall clock period.  The default is Speed = 1 which is to
   deliver at the same rate the media is consumed.  Speed > 1 means that
   the receiver will get content faster than it regularly would consume
   it.  Speed < 1 means that delivery is slower than the regular media
   rate.  Speed values of 0 or lower has no meaning and are not allowed.
   This mechanism enables two general functionalities.  Client side
   scale operations, i.e. the client receives all the frames and makes
   the adjustment to the playback locally.  The second usage is to
   control delivery for buffering of media.  By specifying a speed over
   1.0 the client can build up the amount of playback time it has
   present in its buffers to a level that is sufficient for its needs.

   A naive implementation of Speed would only affect the transmission
   schedule of the media and has a clear impact on the needed bandwidth.
   This would result in the data rate being proportional to the speed
   factor.  Speed = 1.5, i.e. 50% faster than normal delivery, will then
   result in a 50% increase in the data transport rate.  If that can be
   supported or not depends soley on the underlaying network path.
   Scale may also have some impact on the required bandwidth due to the
   manipulation of the content in the new playback schedule.  An example
   is fast forward where only the independently decodable intra frames
   are included in the media stream.  This usage of only intra frames
   increase the data rate significantly compared to a normal sequence
   with the same number of frames where most frames will be encoded
   using prediction.

   This potential increase of the data rate needs to be handled by the
   media sender.  The client has requested that the media is delivered
   in a specific way, which should be honored.  However, the media
   sender can not ignore if the network path between the sender and the
   receiver can't handle the resulting media stream.  In that case the
   media stream needs to be adapted to fit the available resources of
   the path.  This can result in that media quality has be reduced due
   to the delivery modifications that the client has requested.

   The need for bitrate adaptation becomes especially problematic in
   regards to Speed.  If the is target is to fill up the buffer then the
   client may not want to do that at the cost of reduced quality.  If
   you like to do local playout changes then you may actually require
   that the requested speed is honored.  To resolve this issue the usage
   of speed specifies a range so that both usages can be supported.  The
   server is request to use as high as possible speed value within the
   range if the bandwidth is insufficient for the upper bound.  As long
   as the server can maintain a speed value within the range it shall
   not change the media quality, instead modify the speed value in
   response to available bandwidth.  Only if the server becomes unable
   to maintain the lower bound speed value does it need to modify the
   media quality to maintain the lower bound speed value.

   This functionality enables the local scaling implementation to use a
   tight or even a range where lower bound equals upper bound to
   identify that it requries the server to deliver the requested amount
   of media time per delivery time independent of how much it needs to
   adapt the media quality to fit within the available path bandwidth.
   For buffer refilling it is suitable to use a range with a reasonable
   span and with a lower bound at the nominal media rate like 1.0 - 2.5.
   If one likes to reduce the buffer one specifies an upperbound that is
   below 1.0 to force the server to deliver slower than nominal media
   rate.

16.47.  Speed

   The Speed request-header field requests the server to deliver
   specific amounts of nominal media time per unit of delivery time,
   contingent on the server's ability and desire to serve the media
   stream at the given speed.  The client requests the delivery speed to
   be within a given range with an upper and lower bound.  The server
   SHALL delivery at the highest possible speed within the range, but
   not faster than the upper-bound, for which the underlying network
   path can support the resulting transport data rates.  As long as any
   speed value within the given range can be provided the server SHALL
   NOT modify the media quality.  Only if the server is unable to
   delivery media at the speed value provided by the lower bound shall
   it reduce the media quality.

   Implementation of the Speed functionality by the server is OPTIONAL.
   The server can indicate its support through a feature-tag,
   play.scale.  The lack of a Speed header in the response is an
   indication of lack of support of this functionality.

   The speed parameter values are expressed as a positive decimal value,
   e.g., a value of 2.0 indicates that data is to be delivered twice as
   fast as normal.  A speed value of zero is invalid.  The range is
   specified in the form "lower bound - upper bound".  The lower bound
   value may be smaller or equal to the upper bound.  All speeds may not
   be possible to support.  Therefore the server MAY modify the
   requested values to the closest supported.  The actual supported
   speed MUST be included in the response.  Note however that the use
   cases may vary and that Speed value ranges such as 0.7 - 0.8,
   0.3-2.0, 1.0-2.5, 2.5-2.5 all has their usage.

   Example:
     Speed: 1.0 - 2.5

   Use of this header changes the bandwidth used for data delivery.  It
   is meant for use in specific circumstances where delivery of the
   presentation at a higher or lower rate is desired.  The main use
   cases are buffer operations or local scale operations.  Implementors
   should keep in mind that bandwidth for the session may be negotiated
   beforehand (by means other than RTSP), and therefore re-negotiation
   may be necessary.  To perform Speed operations the server needs to
   ensure that the network path can support the resulting bit-rate.
   Thus the media transport needs to support feedback so that the server
   can react and adapt to the available bitrate.

From 20.2.3: Header syntax:

   Speed            =  "Speed" HCOLON POS-FLOAT MINUS POS-FLOAT


   MINUS    =  SWS "-" SWS ; minus/dash



Cheers

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [hidden email]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[hidden email]
https://www.ietf.org/mailman/listinfo/mmusic