SMIL 1.0 solved fundamental media synchronization problems and defined a powerful way of choreographing multimedia content. SMIL Boston extends the timing and synchronization support, adding capabilities to the timing model and associated syntax. This section of the document specifies the Timing and Synchronization module.
There are two intended audiences for this module: implementers of SMIL Boston document viewers or authoring tools, and authors of other XML languages who wish to integrate timing and synchronization support. A language with which this module is integrated is referred to as a host language. A document containing SMIL Timing and Synchronization elements and attributes is referred to as a host document.
@@ Do we still need the next paragraph?
In the process of extending SMIL 1.0 for modularization and use in other XML languages, some alternate syntaxes have been defined. If a document would otherwise be SMIL 1.0 compatible except for use of alternate syntax, the use of the SMIL 1.0 syntax is recommended so the document will be playable by SMIL 1.0 as well as later document players.
As this module is used in different profiles (i.e. host languages), the associated syntax requirements may vary. Differences in syntax should be minimized as much as is practical. The semantics of the timing model and of the associated markup must remain consistent across all profiles. Any host language that includes SMIL Boston Timing and Synchronization markup (either via a hybrid DTD or schema, or via namespace qualified extensions) must preserve the semantics of the model defined in this specification.
Some SMIL 1.0 syntax has been changed or deprecated. Only SMIL document players must support the deprecated SMIL 1.0 attribute names as well as the new SMIL Boston names. A SMIL document player is an application that supports playback of "application/smil" documents (@@ or however we denote SMIL documents as distinct from integration documents).
This section is informative.
SMIL Timing defines elements and attributes to coordinate and synchronize the presentation of media over time. The term media covers a broad range, including discrete media types such as still images, text, and vector graphics, as well as continuous media types that are intrinsically time-based, such as video, audio and animation.
Three synchronization elements support common timing use-cases:
These elements are referred to as time containers. They group their contained children together into coordinated timelines.
SMIL Timing also provides attributes that can be used to specify an element's timing behavior. Elements have a begin, and a simple duration. The begin can be specified in various ways - for example, an element can begin at a time, based upon when another element begins, or when some event (such as a mouse click) happens. The simple duration defines the basic presentation duration of an element. Elements can be defined to repeat the simple duration, a number of times or for an amount of time. The simple duration and any effects of repeat are combined to define the active duration. When an element's active duration has ended, the element can either be removed from the presentation or frozen (held in its final state), e.g. to fill any gaps in the presentation.
@@ Need a simple bar-illustration with labels for the simple and active durations and the frozen state.
The attributes that control these aspects of timing can be applied not only to media elements, but to the time containers as well. This allows, for example, an entire sequence to be repeated, and to be coordinated as a unit with other media and time containers. While authors can specify a particular simple duration for a time container, it is often easier to leave the duration unspecified, in which case the simple duration is defined by the contained child elements. When an element does not specify a simple duration, the time model defines an implicit simple duration for the element. For example, the implicit simple duration of a sequence is based upon the sum of the active durations of all the children.
@@ The last sentence above is imprecise, but vague enough ("based upon") not to be incorrect. This section is not normative, and it is easier to gloss over details to make it flow.
Each time container also imposes certain defaults and constraints upon the contained children. For example in a <seq>, elements begin by default right after the previous element ends, and in all time containers, the active duration of child elements is constrained not to extend past the end of the time container's simple duration.
@@ ??Need a second bar-illustration that shows how a par cuts off children?
The SMIL Timing Model defines how the time container elements and timing attributes are interpreted to construct a time graph. The time graph is a model of the presentation schedule and synchronization relationships. In an ideal environment, the presentation would perform precisely as specified. However, various real-world limitations (such as network delays) can influence the actual playback of media. How the presentation application adapts and manages the the presentation in response to media playback problems is termed runtime synchronization behavior. SMIL includes attributes that allow the author to control the runtime synchronization behavior for a presentation.
The SMIL Timing and Synchronization syntax and precise semantics are described in the following section. A set of symbols are used in the semantic descriptions:
B
d
AD
AE
@@ The next paragraph and list should be moved to a "differences form SMIL 1" section
SMIL 1.0 defines the model for timing, including markup to define element timing, and elements to define parallel and sequence time containers. This version introduces some syntax variations and additional functionality, including:
The complete syntax is described here, including syntax that is unchanged from SMIL 1.0.
This section defines the set of timing attributes that are common to all of the SMIL synchronization elements.
@@ Need to define "local time" or find a different term.
The basic timing for an element is described using the begin
and dur attributes. Authors can specify the begin time of an
element in a variety of ways, ranging from simple clock times to the time
that an event like a mouse-click happens. The simple duration of an element
is specified as a simple time value. The attribute syntax is described below.
@@ ?Need to introduce and better define the notion of syncbase and event base.
<seq> time container.
If no begin is specified, the element begins at the time of
the default syncbase. If there is an error in any individual value in the
list of begin values, only the individual value will be ignored (as though
it were not specified), but the rest of the list will not be invalidated.
If no legal value is specified in the list of begin values, the default value
for begin will be used.
If there is any error in the argument value syntax for dur ,
the attribute will be ignored (as though it were not specified).
@@@ Need to Formalize rules for handling lists of times. Move some of this
to Restarting elements.
Basic rule is to take earliest resolved, valid time.
For begin, all are potentially valid. Earliest begins, and then others begin
or not subject to restart rules.
@@@ Need to discuss, or at least mention how time container constraints interact with specified times. Note that the begin time is just a definition of sync, and especially w.r.t. negative begin times, will be be overridden by the time container constraints. Probably best to just mention it, and point to section within discussion of time containers.
@@@ Need to discuss (somewhere, perhaps not here but in a Details subsection) how to convert a time specified as a syncbase-value (and by extension a wallclock-value or an event in document or system time ) to a time on the parent time container local timeline. Especially given the wallclock stuff, we need to consider the name "local time". Basic mechanism coming down from root is to subtract the begin of a parent, use the remainder after dividing by the simple duration (which may vary over time - yuck!) or subtract the offset of the current repeat iteration from the begin time (better when working on begin resolution on a reset), apply filters for time manipulations (speed, acclerate/decelerate, autoReverse), and then recurse to the parent. Basic mechanism going up reverses all this. Note that the pure conversions do not take into account the clamping of active durations, nor the effects of fill (where time is frozen). An alternate form of the conversion is used when actually sampling the time graph; this clamps times when an element is frozen, to the value of the active end.
If the element does not have a (valid) dur attribute, the simple
duration for the element is defined to be the implicit duration of the element.
The implicit duration depends upon the type of an element. Note that if a
media element has time children (e.g. animate or
area elements), then it is also a <par> time
container. If the media element has no time children, it is described as
a simple media element.
<seq>, <par> and
<excl> time containers, including media elements that
are also time containers, the implicit simple duration is a function of the
children of the time container. For details see the section
Time container durations.
@@ There are problems with making the implicit simple duration of discrete media be indefinite: we need to put in the qualification in seq that an element following an element with indefinite active duration becomes relative to the previous element begin. OTOH, we have to deal with this for other cases as well. But if we make the implicit duration be 0, then we have to add the note that the default for fill magically becomes "freeze". Neither approach is very clean. As an alternative, we could change seq to allow indefinite durations on children, e.g. for event-based end times. With the SMIL 1 semantics we cannot define the children of a sequence to just end on an event - they will all collapse as the default sync-bases get remapped to the previous begin.
If the author specifies a simple duration that is longer than the "intrinsic" defined duration for a continuous media element, the ending state of the media (e.g. the last frame of video) will be shown for the remainder of the simple duration. This only applies to visual media - aural media will simply stop playing.
Note that when the begin attribute refers to an event, or to
the begin or active end of another element, it may not be possible to calculate
when the begin will happen. For example, if an element is defined to begin
on some event, the begin time will not be known until the event happens.
When such a time becomes known (i.e. when it can be calculated as a presentation
time), the time is said to be resolved (see also the discussion
of Unifying scheduled and interactive
timing).
The following example shows simple offset begin timing. The
<audio> element begins 5 seconds after the
<par> time container begins, and ends 4 seconds later.
<par> <audio src="song1.au" begin="5s" dur="4s" /> </par>
The following example shows syncbase begin timing. The
<img> element begins 2 seconds after the
<audio> element begins.
<par> <audio id="song1" src="song1.au" /> <img src="img1.jpg" begin="song1.begin+2s" /> </par>
Elements can also be specified to begin in response to an event. In this example, the image element begins (appears) when the user clicks on element "show". The image will end (disappear) 3 and a half seconds later.
<text id="show" ... /> <img begin="show.click" dur="3.5s" ... />
In the syntax specifications that follow, allowed white space is indicated as "S", defined as follows (taken from the [XML10] definition for "S"):
S ::= (#x20 | #x9 | #xD | #xA)+
A begin-value-list is a semi-colon separated list of timing specifiers:
begin-value-list ::= begin-value (S";"Sbegin-value-list )? begin-value ::= (offset-value | syncbase-value | syncToPrev-value | event-value | media-marker-value | wallclock-sync-value)
An end-value-list is a semi-colon separated list of timing specifiers:
end-value-list ::= end-value (S";"Send-value-list )? end-value ::= (clock-value | syncbase-value | syncToPrev-value | event-value | media-marker-value | wallclock-sync-value)
Several of the timing specification values have a similar syntax. In addition,
XML ID attributes are allowed to contain the dot '.' separator
character. The backslash character '\' can be used to escape the dot separator
in Id and event-name references. To parse an individual item in a value-list,
the following approach defines the correct interpretation.
'+' or '-'), the value should be parsed as an
offset value.
.'
separator characters preceded by a backslash '\' escape character.
.' separator character, then the value
should be parsed as an event-value
with an unspecified (i.e. default) eventbase-element.
.begin" or
".end", then the value should be parsed as a
syncbase-value.
.marker(", then the value
should be parsed as a
media-marker-value.
@@Note that this approach essentially reserves the following tokens:
prev and wallclock for element IDs, and
begin, end and marker for event names.
Should we allow a leading backslash before any of these names to escape them,
or simply reserve these names?
E.g. begin="\wallclock.click; foo.\marker+2s"
Clock values have the following syntax:
Clock-val ::= ( Full-clock-val | Partial-clock-val | Timecount-val )
Full-clock-val ::= Hours ":" Minutes ":" Seconds ("." Fraction)?
Partial-clock-val ::= Minutes ":" Seconds ("." Fraction)?
Timecount-val ::= Timecount ("." Fraction)? (Metric)?
Metric ::= "h" | "min" | "s" | "ms"
Hours ::= DIGIT+; any positive number
Minutes ::= 2DIGIT; range from 00 to 59
Seconds ::= 2DIGIT; range from 00 to 59
Frames ::= 2DIGIT; @@ range?
Subframes ::= 2DIGIT; @@ range?
Fraction ::= DIGIT+
Timecount ::= DIGIT+
2DIGIT ::= DIGIT DIGIT
DIGIT ::= [0-9]
For Timecount values, the default metric suffix is "s" (for seconds). No embedded white space is allowed in clock values, although leading and trailing white space characters will be ignored.
The following are examples of legal clock values:
02:30:03 50:00:10.25 = 50 hours, 10 seconds and 250 milliseconds
02:33 00:10.5 = 10.5 seconds = 10 seconds and 500 milliseconds
3.2h = 3.2 hours = 3 hours and 12
minutes 45min = 45 minutes 30s = 30 seconds 5ms = 5 milliseconds 12.467 = 12 seconds and 467 milliseconds
Fractional values are just (base 10) floating point definitions of seconds.
The number of digits allowed is unlimited (although actual precision may
vary among implementations).
For example:
00.5s = 500 milliseconds
00:00.005 = 5 milliseconds
An offset value has the following syntax:
offset-value ::= ( "+" | "-" )?( Clock-value )
An offset value allows an optional sign on a clock value, and is used to indicate a positive or negative offset.
Note, only compliant SMIL document players are required to support the SMIL 1.0 syncbase-value syntax. Language designers integrating SMIL Boston Timing and Synchronization should not support this syntax.
smil-1-syncbase-value ::= "id(" id-ref ")"
( "(" ( "begin" | "end" | clock-value) ")" )?
ID reference values are references to the value of an "id" attribute of another element in the document.
Id-value ::= IDREF
The IDREF is a legal XML identifier.
A syncbase value has the following syntax:
Syncbase-value ::= ( Syncbase-element "." Time-symbol
)
( S ("+"|"-")
S Clock-value )?
Syncbase-element ::= Id-value
Time-symbol ::= "begin" | "end"
A syncbase value starts with a Syncbase-element term defining the value of an "id" attribute of another element referred to as the syncbase element. The syncbase element must be another timed element contained in the host document. In addition, the syncbase element may not be a descendent of the current element. If the syncbase element specification refers to an illegal element, the syncbase-value description is ignored (although the entire time value list is not invalidated - only the particular syncbase value).
The syncbase element is qualified with one of the following time symbols:
The time symbol can be followed by a clock value. The clock value specifies a presentation time offset from the time (i.e. the begin or active end) specified by the syncbase and time symbol. If the clock value is omitted, it defaults to "0".
No embedded white space is allowed between a syncbase element and a time-symbol. White space will be ignored before and after a "+" or "-" for a clock value. Leading and trailing white space characters (i.e. before and after the entire syncbase value) will be ignored.
Examples:
begin="x.end-5s" : Begin
5 seconds before "x" ends
begin=" x.begin " : Begin when
"x" begins
begin="x.begin + 1m" : End 1 minute after
"x" begins
A sync-to-prev value has the following syntax:
SyncToPrev-value ::= ( "prev." Time-symbol )
(
S ("+"|"-")
S Clock-value )?
A sync-to-prev value is much like a syncbase value, except that the reserved token "prev" is used in place of the Syncbase-element term. The Time-symbol and optional Clock-value offset are as defined for syncbase values.
The previous element is the element that precedes this element within the parent time container (as reflected in the DOM). Note that the parent time container may not be the immediate parent of the current node, in some host documents.
If there is no previous element (i.e. if the current element is the first child of the parent time container), then the begin of the parent time container is used as the syncbase (note that the Time-symbol is ignored in this case). The Clock-value offset is nevertheless added to the parent time container begin time, to yield the resulting time value.
@@Should we also discard the offset? Consider trying to apply a uniform rule like "prev.begin+2s" to a series of elements. For the first one, we really want to make it show up immediately (usually). Would we ever want to preserve the offset in this case?
@@This requires more complete examples, or we need to include them above somewhere. We need good examples of how this is used.
Examples:
begin="prev.end-5s" :
Begin 5 seconds before the previous element ends
begin=" prev.begin " : Begin
when the previous element begins
begin="prev.begin + 1m" : End 1 minute after
the previous element begins
An event value has the following syntax:
Event-value ::= (
Eventbase-element "." )? Event-symbol
( S ("+"|"-")
S Clock-value )?
Eventbase-element ::= Id-value
An Event value starts with an Eventbase-element term that specifies the
event-base element. The event-base element is the element on which
the event is observed. Given DOM event bubbling, the event-base element may
be either the element that raised the event, or it may be an ancestor element
on which the bubbled event can be observed. Refer to DOM-Level2-Events
[DOM2Events] for details.
The "Id-value" is the value of an attribute declared to be an "id" in the
host language, for the event-base element. This element must be another timed
element contained in the host document.
@@ Patrick: I think that the event base need not be timed - consider button clicks in HTML, clicks on regions in SMIL layout docs, etc.
If the Eventbase-element term is missing, the event-base element defaults
to the element on which the attribute is specified (the current element).
If this element has no associated layout (e.g. a time container in a SMIL
document), then some UI events may not be defined (e.g. mouse events). Note
that certain elements may specify a different default eventbase. E.g. the
SMIL Animation elements (animate, animateMotion,
etc.) specify that the default eventbase is the target element of
the animation. See also [SMIL-ANIMATION].
The event value must specify an Event-symbol. This term specifies the name of the event that is raised on the Event-base element. The host language designer must specify which types of events can be used. If an integrating language specifies no supported events, the event-base time value is effectively unsupported for that language.
The last term specifies an optional clock-value that is a presentation time offset from the event. If this term is omitted, the offset is 0.
No embedded white space is allowed between an eventbase element and an event-symbol. White space will be ignored before and after a "+" or "-" for a clock value. Leading and trailing white space characters (i.e. before and after the entire eventbase value) will be ignored.
Note that it is not considered an error to specify an event that cannot be raised on the Event-base element (such as click for audio or other non-visual elements). Since the event will never be raised on the specified element, the event-base value is effectively ignored. Similarly, if the host language allows dynamically created events (as supported by DOM-Level2-Events [DOM2Events]), all possible Event-symbol names cannot be specified, and so unrecognized names may not be considered errors. Host language specifications must include a description of legal event names, and/or allow any name to be used.
The semantics of event-based timing are detailed in the section Unifying Scheduling and Interactive Timing.
Examples:
begin=" x.load " : Begin when "load" is observed
on "x"
begin="x.focus+3s" : Begin 3 seconds after an "focus"
event on "x"
Certain types of media can have associated marker values that associate a name with a particular point (i.e. a time) in the media. The media marker value provides a means of defining a begin or end time in terms of these marker values. Note that if the referenced id is not associated with a media element that supports markers, or if the specified marker name is not defined by the media element, the associated time may never be resolved.
Media-Marker-value ::= Id-value ".marker("
S
marker-symbol S ")" )
@@Should we allow offsets from markers? Is this useful? Note that the offset would be in parent time, not media time.
The marker symbol is a string that must conform to the definition of marker names for the media associated with the Id-value.
Wallclock-sync values have the following syntax. The values allowed are based upon several of the "profiles" described in [DATETIME], which is based upon [ISO8601]. Exactly the components shown here must be present, with exactly this punctuation. Note that the "T" appears literally in the string, to indicate the beginning of the time element, as specified in [ISO8601].
wallclock-val ::= "wallclock(" S (DateTime | WallTime) S ")"
DateTime ::= Date "T" WallTime
Date ::= Years "-" Months "-" Days
WallTime ::= (HHMM-Time | HHMMSS-Time)(TZD)?
HHMM-Time ::= Hours24 ":" Minutes
HHMMSS-Time ::= Hours24 ":" Minutes ":" Seconds ("." Fraction)?
Years ::= 4DIGIT;
Months ::= 2DIGIT; range from 01 to 12
Days ::= 2DIGIT; range from 01 to 31
Hours24 ::= 2DIGIT; range from 00 to 23
4DIGIT ::= DIGIT DIGIT DIGIT DIGIT
TZD ::= "Z" | (("+" | "-") Hours24 ":" Minutes )
Complete date plus hours and minutes: YYYY-MM-DDThh:mmTZD (e.g. 1997-07-16T19:20+01:00) Complete date plus hours, minutes and seconds: YYYY-MM-DDThh:mm:ssTZD (e.g. 1997-07-16T19:20:30+01:00) Complete date plus hours, minutes, seconds and a decimal fraction of a second YYYY-MM-DDThh:mm:ss.sTZD (e.g. 1997-07-16T19:20:30.45+01:00)
Note that the Minutes, Seconds, Fraction, 2DIGIT and DIGIT syntax is as defined for Clock-values. Note that white space is not allowed within the date and time specification.
There are three ways of handling time zone offsets:
No embedded white space is allowed in wallclock values, although leading and trailing white space characters will be ignored.
The presentation engine must be able to convert wallclock-values to a time
within the document. When the document begins, the current wallclock time
must be noted - this is the document wallclock begin. Wallclock
values are then converted to a document time by subtracting the document
wallclock begin, and then converting the time to the element's parent time
space as for any syncbase value, as though the syncbase were the document
body. Note that the resulting begin or end time may be before the begin,
or after end of the parent time container. This is not an error, but the
time container constraints
still apply. In any case, the semantics of the begin and
end attribute govern the interpretation of the wallclock value.
The following examples all specify a begin at midnight on January 1st 2000, UTC
begin="wallclock(2000-01-01Z)" begin="wallclock( 2000-01-01T00:00Z )" begin="wallclock( 2000-01-01T00:00:00Z )" begin="wallclock( 2000-01-01T00:00:00.0Z )" begin="wallclock( 2000-01-01T00:00:00.0Z )" begin="wallclock( 2000-01-01T00:00:00.0-00:00 )"
The following example specifies a begin at 3:30 in the afternoon on July 28th 1990, in the Pacific US time zone:
begin="wallclock( 1990-07-28T15:30-08:00 )"
The following example specifies a begin at 8 in the morning wherever the document is presented:
begin="wallclock( 08:00 )"
New element controls for element time behavior are under discussion. Note that an Accessibility requirement for control of the playback speed is related to (but may end up with different syntax different from) the speed control. In general, these time manipulations are suited to animation and non-linear or discrete media, rather that linear continuous media. Not all continuous media types will support time manipulations, e.g. streaming MPEG 1 video playing backwards.
Support for timing manipulations is not required. Future versions of this module will specify fall-back behavior for user agents that do not support timing manipulations.
These are placed here, as they manipulate the simple duration - they are "inside" the definition of repeating, end, etc.
In the model for time manipulations, the element's local time can be filtered or modified. The filtered time affects all descendents. Any filter that changes the effective play speed of element time may conflict with the basic capabilities of some media players. The use of these filters is not recommended with linear media players, or with time containers that contain linear media elements.
The proposed extensions support use-cases commonly associated with graphic animation.
There are a number of unresolved issues with this kind of time manipulation, including issues related to event-based timing and negative play speeds, as well as many media-related issues.
The speed attribute controls the local playspeed of an element, to speed up or slow down the effective rate of play. Note that the speed does not specify an absolute play speed, but rather modifies the playspeed of the parent time container. Thus if a <par> and one of its children both specify a speed of 50%, the child will play at 25% of normal playspeed. Also note that a speed of -100% is equivalent to playing the media backwards.
Legal values are signed floating point values. Zero values are not allowed.
The default is "1.0" (no modification of speed).
@@ Borrow pictures and description of manipulation from keySplines in SMIL Animation? Need to describe that time is actually manipulated and remapped, but can think of it as progress.
These attributes define a simple acceleration and deceleration of element time, within the simple duration. This is useful for animation, motion paths, etc. The values are expressed as a proportion of the simple duration (i.e. between 0 and 1), and are defined such that the simple duration is not affected (although the normal playspeed is increased to compensate for the periods of acceleration and deceleration). Note that these attributes apply to the simple duration; if these attributes are combined with repeating behavior, the acceleration and/or deceleration occurs within each repeat iteration.
The sum of acceleration and deceleration must not exceed 1. If it does, the deceleration value will be reduced to make the sum legal.
The speed or rate of progress through the simple duration must be increased to account for the acceleration and preserve the simple duration. The adjusted speed is described as the run rate. For an element with both acceleration and deceleration, the speed over the simple duration varies from 0 up to the run rate and then back down to 0.
To compute the run rate over the course of the simple duration, the following
formula is used. Let a be the value of accelerate,
and b be the value of decelerate. The run rate
r is then:
r = 1 / ( 1 - a/2 - b/2 )
Thus, for example, if the value of accelerate is 1 (i.e. accelerate throughout the entire simple duration), the run rate is 2.
The speed s(t) at any point in time
t (within the simple duration
d) is defined as a function of the run rate,
as follows:
For: ( 0 <= t < (a*d) ) I.e. in the acceleration intervals(t)= r * ( t / ( a * d ))
For: ( (a*d) <= t <= (d-(b*d)) ) I.e. in the run-rate interval
s(t) = r
For: ( (d-(b*d)) < t <=d) I.e. in the deceleration intervals(t)= r * ( ( t - (d-(b*d)) ) / ( b*d ))
If in place of t we use
p, the proportional progress through the simple
duration, the equations simplify somewhat:
p=t/d
For: ( 0 <= p < a ) I.e. in the acceleration intervals(p)= r * ( p / a )
For: ( a <= p <= (1-b) ) I.e. in the run-rate interval
s(p) = r
For: ( (1-b) < p <= 1 ) I.e. in the deceleration intervals(p)= r * ( ( p - (1-b) ) / b )
In this example, a motion path will accelerate up from a standstill over
the first 2 seconds, run at a faster than normal rate for 4 seconds, and
then decelerate smoothly to a stop during the last 2 seconds. This
makes an animation look more realistic. The animateMotion element
is defined in the Animation section of SMIL Boston.
<img ...>
<animateMotion dur="8s" accelerate=".25" decelerate=".25" .../>
</img>
In this example, the image will "fly in" from offscreen left , and then
decelerate quickly during the last second to "ease in" to place. This assumes
a layout model that supports positioning (a similar effect could be achieved
by animation the position of a region in SMIL layout). The
animate element is defined in the Animation section of SMIL
Boston.
<img ...>
<animate attributeName="left" dur="4s" decelerate=".25"
from="-1000" to="0" additive="sum" />
</img>
This defines "play forwards then backwards" functionality. The use of autoReverse effectively doubles the simple duration. When combined with repeating behavior, each repeat iteration will play once forwards, and once backwards. This is useful for animation, especially for mechanical and pendulum motion.
Argument values are Booleans.
The default value is false (i.e. play normally).
SMIL 1.0 introduced the repeat attribute, which is used to repeat a media element or an entire time container. SMIL Boston introduces two new controls for repeat functionality that supercede the SMIL 1.0 repeat attribute. The new attributes, repeatCount and repeatDur, provide a semantic that more closely matches typical use-cases, and the new attributes provide more control over the duration of the repeating behavior. The SMIL 1.0 repeat attribute is deprecated in SMIL Boston (it must be supported in SMIL document players for backwards compatibility).
Repeating an element causes the simple duration to be "played" several times
in sequence. This will effectively copy or loop the contents of
the element media (or an entire timeline in the case of a time container).
The author can specify either how many times to repeat, using
repeatCount, or how long to repeat, using
repeatDur. Each repeat iteration is one instance of
"playing" the simple duration.
If the simple duration is indefinite, the element cannot repeat. See also the sections Computing the Active Duration and Propagating Changes to Times.
At most one of repeatCount or repeatDur should
be specified. If both are specified (and the simple duration is not indefinite),
the active duration is defined as the minimum of the specified repeatDur,
and the simple duration multiplied by repeatCount. For the purposes
of this comparison, a defined value is considered to be "less than" a value
of "indefinite". If the simple duration is indefinite, and both
repeatCount or repeatDur are specified, the
repeatCount will be ignored, and the repeatDur
will be used (refer to the examples below describing repeatDur
and an indefinite simple duration). These rules are included in the section
Computing the Active Duration.
Need to create normative examples that demonstrate the new controls, and the interaction with implicit and explicit simple durations. Examples must also demonstrate the interaction of repeating behavior and time container constraints.
@@ Need to add example of repeatCount < 1 and/or repeatDur < simple duration
In the following example, the 2.5 second simple duration will be repeated twice; the active duration will be 5 seconds.
<audio src="background.au" dur="2.5s" repeatCount="2" />
In the following example, the 3 second simple duration will be repeated two full times and then the first half is repeated once more; the active duration will be 7.5 seconds.
<audio src="background.au" dur="3s" repeatCount="2.5" />
In the following example, the audio will repeat for a total of 7 seconds. It will play fully two times, followed by a fractional part of 2 seconds. This is equivalent to a repeatCount of 2.8.
<audio src="music.mp3" dur="2.5s" repeatDur="7s" />
Note that if the simple duration is zero (0) or indefinite, repeat behavior
is not defined (but repeatDur still defines the active duration).
In the following example the simple duration is indefinite, and so the
repeatCount is effectively ignored. Nevertheless, this is not
considered an error. The active duration is also indefinite.
<img src="foo.png" dur="indefinite" repeatCount="2" />
In the following example, the simple duration is 0, and so repeat behavior
is not meaningful. However, the repeatDur still determines the
active duration. The effect is the text is shown for 10 seconds.
<text src="intro.html" repeatDur="10s" />
In the following example, if the audio media is longer than the 5 second
repeatDur, then the active duration will effectively cut short
the simple duration.
<audio src="sound.au" repeatDur="5s" />
The repeatCount and repeatDur attributes can also
be used to repeat an entire timeline (i.e. a time container simple duration),
as in the following example. The sequence has an implicit simple duration
of 13 seconds. It will begin to play after 5 seconds, and then will
repeat the sequence of three images 3 times. The active duration is thus
39 seconds long.
<seq begin="5s" repeatCount="3" > <img src="img1.jpg" dur="5s" /> <img src="img2.jpg" dur="4s" /> <img src="img3.jpg" dur="4s" /> </seq>
The SMIL 1.0 repeat attribute behaves in a manner similar to repeatCount, but it defines the functionality in terms of a sequence that contains the specified number of copies of the element without the repeat attribute. This definition has caused some confusion among authors and implementers. See also the SMIL 1.0 specification [SMIL10].
In particular, there has been confusion concerning the behavior of the SMIL
1.0 end attribute when used in conjunction with the
repeat attribute. SMIL Boston complies with the common practice
of having the end attribute define the element's simple duration
when the deprecated repeat attribute is used. Only SMIL document
players must support this semantic for the end attribute. Only
a single SMIL 1.0 "end" value (i.e. an
offset-value or a
smil-1.0-syncbase-value,
but none of the new SMIL Boston timing) is permitted when used with the
deprecated repeat attribute. If repeat is used
with repeatCount or repeatDur on an element, or
if repeat is used with an illegal end value, the
repeat value is ignored.
<seq> element with the stated number
of copies of the element without the "repeat" attribute as children. All
other attributes of the element, including any begin delay, are included
in the copies.
SMIL Boston provides an additional control over the active duration. The
end attribute allows the author to specify the active end of
the element using a simple offset, a syncbase, an event-base or DOM methods
calls.
If end specifies an event-value or syncbase-value that is not
resolved, the active duration is indefinite (until resolved).
The active duration is defined by the simple duration, the repeat attributes
and end. The rules for combining these are presented in the
section Computing the Active Duration.
@@@ Need to Formalize rules for handling lists of end times.
Basic rule is to take earliest resolved, valid time, that is not in the past.
@@Need example with end cutting short a longer simpler dur might also be good.
In the following example, the active duration will end at the earlier of 10 seconds, or the end of the "foo" element. This is particularly useful if "foo" is defined to begin or end relative to an event.
<audio src="foo.au" dur="2s" repeatDur="10s"
end="foo.end" .../>
In the following example, the element begins when the user clicks on the "gobtn" element. The active duration will end 30 seconds after the parent time container begins. Note that if the user has not clicked on the target element before 30 seconds elapse, the element will never begin.
<par> <audio src="music.au" begin="gobtn.click" repeatDur="indefinite" end="30s" ... /> </par>
The defaults for the event syntax make it easy to define simple interactive behavior. The following example stops the video when the user clicks on the element.
<video src="video.mpg" dur="indefinite" end="click" />
Using end with an event value enables authors to end an element
based on either an interactive event or a maximum active duration. This is
sometimes known as lazy interaction.
In this example, a presentation describes factory processes. Each step is a video, and set to repeat 3 times to make the point clear. Each element can also be ended by clicking on some element "next" that indicates to the user that the next step should be shown.
<par ... >
<video id="step1" begin="0" dur="5s"
repeatCount="3" end="nextBtn.click" src.../>
<video id="step2" begin="step1.end" dur="5s"
repeatCount="3" end="nextBtn.click" src.../>
<video id="step3" begin="step2.end" dur="5s"
repeatCount="3" end="nextBtn.click" src.../>
<video id="step4" begin="step3.end" dur="5s"
repeatCount="3" end="nextBtn.click" src.../>
<video id="step5" begin="step4.end" dur="5s"
repeatCount="3" end="nextBtn.click" src.../>
</par>
In this case, the active end of each element is defined to be the earlier of 15 (5s dur * 3 repeats) seconds after it begins, or a click on "next". This lets the viewer sit back and watch, or advance the presentation at a faster pace.
The table in Figure @@@ shows the semantics of all possible combinations
of simple duration, repeatCount and repeatDur,
and end.
Note that while the active duration is computed according to the rules in the table, the parent time container places constraints upon the active duration of all children. These constraints may cut short the active duration of any child, and so override the definition described here. For more information, see the section Time Container constraints on child durations.
In the table below, if a cell is empty, it indicates that the associated
attribute is omitted in the syntax. Where the table entry is "defined", this
should be interpreted as a specified or implicit value other than
"indefinite". Where the entry is a star ("*"), the value does not matter
and can be any of the possibilities. Note that where the active duration
is specified as the minimum of several values (MIN), it may not always be
possible to calculate this when the document begins. If the end
is event-based or DOM-based, then an event or method call that activates
end before the duration specified by
repeatCount or repeatDur will cut short the active
duration at the end activation time.
Note that if either the simple duration or the value of end
cannot be resolved, the respective value is considered to be "indefinite"
for the purposes of evaluating the active duration. If and when the respective
value becomes resolved, the active duration is reevaluated.
Note that for any active duration and simple duration that are both not
indefinite, the number of repeat iterations is defined by the active duration
divided by the simple duration (this may yield partial repeat iterations,
just as repeatCount specifies).
If both end and either (or both) of repeatCount
or repeatDur are specified, the active duration is defined by
the minimum duration defined by the respective attributes.
If end is specified but neither of repeatCount
or repeatDur are specified, then the active duration is defined
as the minimum of the simple duration and the duration defined by
end.
Note also that it is possible to have an indefinite simple duration and a defined, finite active duration. The active duration can cut short the simple duration, but the active duration does not define the simple duration, or change its value.
The following symbols are used in the table as a shorthand:
B
d
| Simple duration d |
repeatCount |
repeatDur |
end |
Active Duration |
| defined | d | |||
| defined | defined | repeatCount*d |
||
| defined | defined | repeatDur |
||
| defined | defined | MIN( d, end-B ) |
||
| defined | defined | defined | MIN( repeatCount*d, repeatDur
) |
|
| defined | defined | defined | MIN( repeatCount*d,
( end-B )) |
|
| defined | defined | defined | MIN( repeatDur,
( end-B )) |
|
| defined | defined | defined | defined | MIN( repeatCount*d,
repeatDur,
( end-B )) |
| indefinite | * | indefinite | ||
| indefinite | * | defined | repeatDur |
|
| indefinite | * | defined | end-B |
|
| indefinite | * | defined | defined | MIN( repeatDur,
( end-B )) |
| * | indefinite | indefinite | ||
| * | indefinite | indefinite | ||
| * | indefinite | indefinite | indefinite | |
| * | indefinite | defined | end-B |
|
| * | indefinite | defined | end-B |
|
| * | indefinite | indefinite | defined | end-B |
Fig. @@@: Computing the active duration for different combinations of the
simple duration, repeatDur and repeatCount, and
end.
It is possible to combine scheduled and interactive timing, e.g.:
<par dur="30s"> <img id="mutebutton" src="mute.jpg"/> <text src="description.html" /> <audio src="audio.au" end="mutebutton.click"/> </par>
The image and the text appear for the specified duration of the
<par> (30 seconds). The audio will stop early if the image is
clicked before the active end of the audio (which in this case is the duration
of the actual audio media "audio.au").
It is possible to declare both a scheduled duration, as well as an event-based active end. This facilitates what are sometimes called "lazy interaction" use-cases, such as a slideshow that will advance on its own, or in response to user clicks:
<seq> <img src="slide1.jpg" dur="10s" end="click" /> <img src="slide2.jpg" dur="10s" end="click" /> <img src="slide3.jpg" dur="10s" end="click" /> <!-- etc., etc. --> </seq>
In this case, the active end of each element is defined to be the earlier of the specified duration, or a click on the element. This lets the viewer sit back and watch, or advance the slides at a faster pace.
By default when an element's active duration ends, it is no longer presented (or its effect is removed from the presentation, depending upon the type of element). Freezing an element extends it, using the last state presented in the active duration. This can be used to fill gaps in a presentation, or to extend an element as context in the presentation (e.g. with additive animation - see [SMIL-ANIMATION]).
The fill attribute allows an author to specify that an element
should be extended beyond the active duration by freezing the final
state of the element. For discrete media, the media is simply displayed as
it would be during the active duration. For continuous media, the "frame"
that corresponds to the end of the active duration is shown. For algorithmic
media like animation, the value defined for the end of the active duration
should be used. The syntax of the fill attribute is the same as in
SMIL 1.0, with one extension:
This attribute only has an effect on visual media elements. Non-visual media elements (audio) should ignore this.
Note that <area> hyperlinks defined as children of a media
element are still active during fill="freeze". See also the SMIL 1.0
specification [SMIL10].
An element with fill="freeze" is extended according to the parent
time container:
<par>, the element is frozen to extend to the end
of the simple duration of the <par>. In this case,
fill="freeze" is equivalent to fill="hold".
<seq>, the element is frozen to extend to the begin
of the next element in the <seq>. This will fill any gap
in the presentation (although it may have no effect if the next element begins
immediately).
<excl>, the element is frozen to extend to the begin
of the next element to be activated in the <excl>. This
will fill any gap in the presentation (although it may have no effect if
the next element interrupts the current element). Note that if an element
is paused, the active duration has not ended, and so the fill
attribute does not (yet) apply. See also the section
The excl time container.
The fill attribute can be used to maintain the value of an media
element after the active duration of the element ends:
<par endSync="last"> <video src="intro.mpg" begin= "5s" dur="30s" fill="freeze" /> <audio src="intro.au" begin= "2s" dur="40s"/> </par>
The video element ends 35 seconds after the parent time container began, but the video frame at 30 seconds remains displayed until the audio element ends. The attribute "freezes" the last value of the element for the remainder of the time container's simple duration.
This functionality is also useful to keep prior elements on the screen while
the next item of a <seq> time container prepares to display
as in this example:
<seq> <video id="v1" fill="freeze" src.../> <video id="v2" begin="2s" src.../> </seq>
The first video is displayed and then the last frame is frozen for 2 seconds, until the next element begins. Note that if it takes additional time to download or buffer video "v2" for playback, the first video "v1" will remain frozen until video "v2" actually begins.
When an element is defined to begin at a simple offset (e.g.
begin="5s" ), there is an unequivocal time when the element
begins. However, if an element is defined to begin relative to an event (e.g.
begin="foo.click" ), the event can happen at any time, and moreover
can happen more than once (e.g. if the user clicks on "foo" several
times). In some cases, it is desirable to restart an element if
a second begin event is received. In other cases, an author may want to preclude
this behavior.
In SMIL Boston, an element can have a list of begin values. In some cases, the intent is to begin at the earliest of the specified times (e.g. when the user clicks on any one of several images). In other cases, the intent is that the element restart when any of the begin times is encountered.
In addition, if an element is defined to begin relative to when another element
begins (using the syncbase- value syntax), the syncbase element can restart.
The restart attribute is used to control the restart behavior
of an element.
The default value for the restart attribute is "always", this may not be a sensible default in all documents, in particular SMIL Boston documents with streaming media would likely want restart="never" set on all of the elements. In order to not require restart="never" be added to every media element in the document, the WG is investigating defining a method to override the default and set a new default for the document. Possibilities under consideration are a <meta> element in the head and an attribute on the <body> element. Language designers could specify other mechanisms for languages that do not have a <body> element.
Note that there are several ways that an element may be restarted. The behavior
(i.e. to restart or not) in all cases is controlled by the
restart attribute. The different restart cases are:
begin specified as an event-value can be restarted
when the named event fires multiple times.
begin specified as a syncbase value, where the
syncbase element can restart. When an element restarts, other elements defined
to begin relative to the begin or active end of the restarting element will
also restart (subject to the value of restart on these elements).
begin specified as "indefinite" can be restarted
when the DOM "beginElement()" method is called repeatedly.
begin specified as "indefinite" can be restarted
by repeatedly actuating a hyperlink that has the element as its target.
When an element restarts, the primary semantic is that it behaves as though this were the first time the element had begun, independent of any earlier behavior. Any effect of an element playing earlier is no longer applied, and only the current begin "instance" of the element is reflected in the presentation.
Note that if the parent time container (or any ascendant time container)
repeats or restarts, any state associated with restart="never"
will be reset, and the element can begin again normally.
The restart setting for an element is evaluated not when the syncbase element restarts or when the eventbase event happens, but rather when the element would actually restart (i.e. taking into account any offset from the syncbase or eventbase). For example:
<img id="go_btn" fill="freeze" .../>
<video id="foo" begin="go_btn.click" ..." />
<audio id="bar" begin="foo.begin+2s" dur="10s"
restart="whenNotActive" ..." />
If the user clicks on the "go_btn" image at 5 seconds, element "foo" will
begin, and element "bar" will be scheduled to begin at 7 seconds. If the
user clicks the image again at 6 seconds, "foo" would restart. The effective
restart of "bar" is scheduled for 2 seconds later, at 8 seconds. However,
since "bar" will begin playing at 7 seconds, the restart at 8 seconds will
be ignored (the restart attribute is set to only allow restart
when "bar" is not active). Element "bar" plays from 7 seconds until 17 seconds.
If the user again clicks the image at 16 seconds, "foo" will restart. Although
"bar" is active when the user clicks the image, the restart for "bar" is
not scheduled until 2 seconds later at 18 seconds. At this point, "bar" will
have ended the first time, and so can legally restart.
The same rules apply if an element is defined to begin when another ends, as in this example:
<img id="go_btn" fill="freeze" .../>
<video id="foo" begin="go_btn.click" dur="2s" .../>
<audio id="bar" begin="foo.end" dur="10s"
restart="whenNotActive" ..." />
If the user clicks on the "go_btn" image once at 5 seconds, "foo" will play from 5 to 7 seconds, and "bar" will play from 7 to 17 seconds. If the user clicks again at 16 seconds, "foo" will restart and play again from 16 to 18 seconds. At 18 seconds, "bar" will have ended, and so will restart and play from 18 to 28 seconds.
@@ this seems useful to point out but a SMIL element cannot be visible before it begins so having a begin="click" means it won't ever begin.
@@Patrick: except that the begin may not mean make it visible - e.g. with timeAction, it could mean apply a class or style, and so begin=click is reasonable.
A common use-case requires that the same UI event is used begin an element
and to end the active duration of the element. This is sometimes described
as "toggle" activation, because the UI event toggles the element "on" and
"off". The restart attribute can be used to author this,
as follows:
<img id="foo" begin="bar.click" end="bar.click"
restart="whenNotActive" ... />
If "foo" were defined with the default restart behavior "always", a second
click on the "bar" element would simply restart the element. However, since
the second click cannot restart the element when restart is
set to "whenNotActive", the element ignores the "begin" specification of
the "click" event. The element can then use the "click" event to end the
active duration and stop the element.
This is based upon the event sensitivity semantics described in Unifying Scheduling and Interactive Timing.
SMIL Boston specifies three time containers: <par>, <seq>, and <excl>.
<par> container defines a simple parallel time grouping
in which multiple elements can play back at the same time.
The default syncbase of the child elements of a <par>
is the begin of the <par>. This is the same element
introduced with SMIL 1.0.
The <par> element supports all element timing.
<seq> container defines a sequence of elements in which
elements play one after the other.
This is the same element introduced with SMIL 1.0, but the semantics (and allowed syntax) for child elements of a <seq> are clarified. The syncbase of a child element is the active end of the previous element. Previous means the element that occurs before this element in the sequence time container. For the first child of a sequence (i.e. where no previous sibling exists), the syncbase is the begin of the sequence time container.
Child elements may define an offset from the syncbase, but may not define
a different syncbase (i.e. they may not define a begin time relative to another
element, or to an event). Child elements may define an
end.
Thus for children of a sequence, only the offset begin values are legal. None of the following begin values may be used:
No constraints are placed upon the end argument values.
The <seq> element itself supports all element timing.
SMIL Boston defines a new time container, <excl>.
The default syncbase of the child elements of the
<excl> is indefinite (i.e. equivalent to
begin="indefinite").
The <excl> element itself supports all element timing.
With the <excl> time container, common use cases that
were either difficult, or impossible, to author are now easier and possible
to create. The <excl> time container is used to define
a mutually exclusive set of clips, and to describe pausing and resuming behaviors
among these clips. Examples include:
The interactive playlist use case above could be accomplished using a
<par> whose sources have interactive begin times and
end events for all other sources. This would require a prohibitively
long list values for end to maintain. The
<excl> time container provides a convenient short hand
for this - the element begin times are still interactive, but the
end events do not need to be specified because the
<excl>, by definition, only allows one child element to
play at a time.
The audio descriptions use case is not possible without the pause/resume
behavior provided by <excl>. This use case would be authored
with a video and each audio description as children of the
<excl>. The video element would be scheduled to begin
when the <excl> begins and the audio descriptions would
start at scheduled begin times or in response to stream events raised at
specific times.
The dynamic video sub-titles use case requires the "play only one at a time"
behavior of <excl>. In addition, the child elements are
declared in such as way so to preserve the sync relationship to the video:
<par>
<video id="vid" .../>
<excl>
<par begin="englishBtn.click" >
<audio begin="vid1.begin" src="english.au" />
</par>
<par begin="frenchBtn.click" >
<audio begin="vid1.begin" src="french.au" />
</par>
<par begin="swahiliBtn.click" >
<audio begin="vid1.begin" src="swahili.au" />
</par>
</excl>
</par>
The three <par> elements are children of the
<excl>, and so only one can play at a time. The audio
child in each <par> is defined to begin when the video
begins. Each audio can only be active when the parent time container
(<par>) is active, but the begin still specifies the
synchronization relationship. This means that when each
<par> begins, the audio will start playing at some point
in the middle of the audio clip, and in sync with the video.
The <excl> time container is useful in many authoring
scenarios by providing a declarative means of describing complex clip
interactions.
@@ Note that the specifics of the pause functionality and how it is represented in syntax are still under discussion, and may well change.
As in the audio descriptions use case above, it is possible to specify whether
a child of the <excl> should pause or stop the clip that
it is interrupting. This is done be specifying the whenDone
attribute on the children of the <excl>. The default behavior
of whenDone is to stop the currently playing clip. The
whenDone attribute only applies to children of an
<excl>. Children of an <excl> that
have been paused are placed on a stack and playback is resumed
for elements in the stack in last-in, first-out order.
<excl> time
containers.<excl>.
@@Need to define how recursion stops: if an element is begun that is on the stack, the stack is popped all the way down to (and including) the newly begun element. This may seem harsh, but it cleanly prevents odd recursion on the stack.
@@Need to resolve whether we want to use whenDone on the interruptor, or interrupt attribute on the interruptee, or some other variant.
In the following example, "default_audio.ra" begins playing when the
<excl> begins. If the user clicks on the intro button,
"intro_audio.ra" begins and "default_audio.ra" is paused because the intro
audio's whenDone attribute is set to "resume". If the user then
selects the moreinfo button, "intro_audio.ra" is paused and "moreinfo_audio.ra"
begins. When the active duration of "moreinfo_audio.ra" ends, "intro_audio.ra"
resumes from where it was paused. When the active duration of "intro_audio.ra"
ends, "default_audio.ra" resumes from where it was paused. When the active
duration of "default_audio.ra" ends, the simple duration of the
<excl> ends.
<excl>
<audio src="default_audio.ra" begin="0" .../>
<audio src="intro_audio.ra" begin="intro.click"
whenDone="resume" .../>
<audio src="moreinfo_audio.ra" begin="moreinfo.click"
whenDone="resume" .../>
</excl>
<excl>
Although the default begin value for children of an
<excl> is indefinite, scheduled begin times are permitted.
Scheduled begin times on children of the <excl> cause
the element to begin at the specified time, pausing or stopping other siblings
depending on the value of the whenDone attribute.
To specify that a child of the <excl> should begin playing
by default (i.e., when the <excl> begins), specify
begin="0" on that child element. If children of an
<excl> are scheduled to begin at the same time, the evaluation
proceeds in document order, and so the last of these elements in document
order will end up being played. The same semantics is applied if multiple
children of the <excl> specify the same begin event-base
value. Nevertheless, if the whenDone attribute of children sharing
begin times or events is set to "resume", the elements that occur earlier
in the document will be paused. In this case, the children will play back
in the reverse of the document order, as the pause stack "unwinds".
In the following slideshow example, images begin at the earlier of their scheduled begin time or when activated by a user input event:
<excl> <img src="image1.jpg" begin="0s".../> <img src="image2.jpg" begin="10s; image1.click".../> <img src="image3.jpg" begin="20s; image2.click".../> </excl>
Note, some surprising results may occur when combining scheduled and interactive
timing within an <excl>. If in the above example, the
user clicks on image1 and then on image2 before ten seconds have elapsed,
image 2 will re-appear at the ten second mark. Image 3 will appear at twenty
seconds.
Children of the <excl> can be activated by hyperlinks
or by events. When using events, the <excl> time container
must be active for child elements of the <excl> to be
activated by an event. When using hyperlinks, if the
<excl> is not active, a seek will occur to the begin time
of the <excl> and the children of the element will be
activated. If the <excl> is currently active when the
hyperlink is selected, a seek does not occur and playback of the child element
begins. Playback of other active elements outside the scope of the
<excl> is unaffected.
When seeking to a child of the <excl>, normal activation
rules apply. If activation of a child of the <excl> requires
advancing the timeline forward, the document should be in a state identical
to that as if the document presentation time advanced undisturbed to reach
the seek time.
<par> and
<excl> time containers. It controls the end of the simple duration
of these containers, as a function of the children. This is particularly
useful with children that have "unknown" duration, e.g. an mpeg movie, that
must be played through to determine the duration.
<par> or <excl> simple duration
ends with the earliest active end of all the child elements. This does not
refer to the lexical first child, or to the first child to start, but rather
refers to the first child to end. <par> or <excl> simple duration
ends when all of the child elements have ended their respective active durations.
Elements with indefinite or unresolved begin times will keep the
simple duration of the time container from ending. <par> or <excl> ends with the
specified child. The id must correspond to one of the immediate children
of the <par> time container. <par ... endSync="movie1" ...>
Semantics of endSync and indeterminate children:
endSync="first" means that the time container must wait for
any element to actually end its active duration. It does not matter whether
the the first element to end was scheduled or interactive.endSync="last" means that the time container must wait for all
elements that have a resolved begin, to end the respective active durations.
Note that elements that had an interactive begin, but that became resolved
before all scheduled elements ended, are added to the set of children that
must end their active durations before the parent can end. This can chain,
so that only one element is running at one point, but before it ends its
active duration another interactive element is resolved. It may even yield
"dead time" (where nothing is playing), if the resolved begin is
after the other elements active end. <excl> that are currently paused
and waiting to resume will keep the simple duration of the time
container from ending.endSync="all" means that every child element of the time container
must end the active duration. Elements do not have to play to completion,
but must have played at least once. When no elements are active, and when
all elements have run one or more times, then the parent can end. endSync=[id-ref] means that the time container must
wait for the referenced element to actually end its active duration. The
id-ref must refer to a child of the time container. If the referenced child
has an indefinite active duration, then the simple duration of the time container
is also indefinite.The implicit duration of a time container is defined in terms of the children of the container. The children can be thought of as the "media" that is "played" by the time container element. The semantics are specific to each of the defined time container variants.
By default, the simple duration of a <par> is defined
by the endSync=last semantics. The simple duration will end
when all scheduled children have ended their respective active durations.
The simple duration of a <par> container can be controlled
with the dur and endSync attributes. If the
dur attribute is specified, the endSync attribute
is ignored. Using endSync, the end of the simple duration can be tied to
the active end of the first child that finishes, or to the active end of
the last child to finish (the default), or to the active end of a particular
child element.
By default, the simple duration of a <seq> ends with
the active end of the last child of the <seq>.
If the last child of a <seq> has an indefinite active
duration, the simple duration of the <seq> is also
indefinite.
The implicit simple duration of an <excl> container is
defined the same as for a <par> container, using the
endSync=last semantics. However, since the default timing for
children of <excl> is interactive, it will be common for
<excl> time containers to have indefinite simple duration.
For endSync={last or all}: The time children and the intrinsic
media duration define the simple duration of the media element time container.
If a continuous media duration is longer than the extent of all the time
children, the continuous media duration defines the implicit simple duration
for the media element time container. If the media is discrete, this is defined
as for <par> elements.
For endSync={first}: The time children and the intrinsic media
duration define the simple duration of the media element time container.
The element ends when the first active duration ends, as defined above for
endSync on a <par>, but no sooner than the
end of the intrinsic media duration of continuous media. If the media is
discrete, this is defined as for <par> elements.
For endSync={ID}: This is defined as for
<par> elements.
If the calculated implicit simple duration is longer than the intrinsic duration for a continuous media element, the ending state of the media (e.g. the last frame of video) will be shown for the remainder of the simple duration. This only applies to visual media - aural media will simply stop playing.
Time containers place certain overriding constraints upon the child elements. These constraints can cut short the active duration of any child element.
All time containers share the basic overriding constraint:
While the child may define a sync relationship that places the begin before the parent begin, the child is not active until the parent begins.
If the child defines an active duration (or by the same token a simple duration) that extends beyond the end of the parent simple duration, the active duration of the child will be cut short when the parent simple duration ends. Note that this does not imply that the child duration is automatically shortened, or that the parent simple duration is "inherited" by the child.
For example:
<par dur="10s" repeatDur="25s"> <video dur="6s" repeatCount="2" .../> <text begin="5s" dur="indefinite" .../> <audio begin="prev.end" .../> </par>
The video will play once for 6 seconds, and then a second time but only for
4 seconds - the last 2 seconds will get cut short and will not be seen. The
text shows up for the last 5 seconds of the <par>, and
the indefinite duration is cut short at the end of the simple duration of
the <par>. The audio will not show up at all, since it
is defined to begin at the end of the active duration of the previous element
(the <text> element). Since the text element ends when
the time container ends, the audio would begin after the time container has
ended, and so never is heard. When the <par> repeats the first time,
everything has happens just as it did the first time. However the last repeat
is only a partial repeat (5 seconds), and so on the video will be seen, but
it will not be seen to repeat, and the last second of the video will be cut
off.
Note the time container is itself subject to the same constraints, and so may be cut short by some ascendant time container. When this happens, the children of the time container are also cut off, in the same manner as for the last partial repeat in the example above.
In addition, <excl> time containers allow only one child
to play at once. Subject to the semantics of the whenDone attribute,
the active duration of an element may be cut short when another element in
the time container begins.
Probably need to define "active" and "inactive" more formally.
We need a few good examples to illustrate these concepts.
SMIL 1.0 defined constraints on sync-arc definition (e.g., begin="image1.begin"), allowing references only to qualified siblings. SMIL Boston explicitly removes this constraint. SMIL Boston also adds event-based timing. Both sync-arcs and event-timing are constrained by the parent time container of the associated element as described above.
While a sync-arc is explicitly defined relative to a particular element, if this element is not a sibling element, then the sync is resolved as a sync-relationship to the parent. If the defined sync would place the element effective begin before the parent time container begin, part of the element will simply be cut off when it first plays. This is not unlike the behavior obtained using clipBegin. However unlike with clipBegin, if the sync-arc defined child element also has repeat specified, only the first iteration will be cut off, and subsequent repeat iterations will play normally.
Note that in particular, an element defined with a sync-arc begin will not automatically force the parent or any ancestor time container to begin.
For the case that an element with a sync-arc is in a parent (or ancestor) time container that repeats: for each iteration of the parent or ancestor, the element is played as though it were the first time the parent timeline was playing. This may require a reset of some sort in the implementation to ensure that the sync relationship to the parent time container is recalculated.
The parent time container must be active for the child element to receive events.
Sequence children are only sensitive to events between the active end of the previous element and the active begin of the following element. Example: all children listen to same begin event and it works:
<seq> <img src="img1.jpg" begin="foo.click" dur="5s" fill="freeze"/> <img src="img2.jpg" begin="foo.click" dur="5s" fill="freeze"/> <img src="img3.jpg" begin="foo.click" dur="5s" fill="freeze"/> </seq>
A negative begin (delay) value defines a clipBegin for the first -- and only the first -- iteration of a repeated element. Without specifying a repeat attribute, a negative begin value and clipBegin are synonymous.
At any moment in time, a timed element is in exactly one of the following states: idle, active, finished or frozen. The state transitions are caused by events called start, restart, freeze and stop. Figure @@ shows the legal transitions between the states of an element:
Fig @@@: State diagram of an element
The following sections explain the semantics of the states and transitions of a timed element, and explain how to define the state transitions using timing attributes of the element.
Note that the states and transitions are part of the model, and do not imply a particular implementation. Note also that an element may transition through more than one state in a virtual instant (i.e. with no time spent in a given state).
The presentation effect of timed elements is generally to display media, or to play a timeline (e.g. for time containers). In some cases, the element may be an animation that manipulates the presentation, but does not directly display anything. In some integration scenarios, the presentation effect of the element may be to apply a stylesheet, or to otherwise modify the presentation. In these discussions, the common case of displaying media or playing a timeline is used to describe the states and transitions. The same semantics should be understood to apply to all defined actions or presentation effects, as specified in the language that integrates SMIL Timing and Synchronization.
@@ We need to note that when the parent (or any ascendant) time container repeats or restarts, the element implicitly is reset to the "idle" state.
When the document that contains a timed element is first presented, the element is created in the idle state. This is the common starting state for all timed elements.
In the idle state a timed element is inactive and does not affect
the presentation of the document in any way. The element simply waits for
the time or event specified in its begin attribute. Note that
the element may transition immediately to the active state if the element
begins immediately when the document begins.
For an element to become active, the element's parent time container must
be active. Given this, a timed element in the idle state
transitions to the active state when the condition specified in
the begin attribute becomes true. As described in the section
on the begin attribute, this condition may depend upon one of
several factors:
Additionally, an element may be started by a DOM beginElement()
method call, or if the user activates (traverses) a hyperlink that is targeted
at the element.
An element may become active as soon as its parent time container becomes
active, if the condition specified in the begin attribute is
true at that point.
Note that the begin attribute can specify a condition that is
a list of values. The specific semantics of evaluating the list of values
is described in the section Basic - begin and
dur.
In the active state, a timed element displays the associated media
or performs the described timeline associated with the element. The active
state includes the entire active duration of the element. The active duration
of an element is specified by the interaction between the dur,
end, repeatDur, and repeatCount attributes
as detailed in the section Computing
the Active Duration.
If a timed element has the fill attribute set to "freeze"
or "hold", upon reaching the end of its active duration, the element will
transition to the frozen state.
In the frozen state the element will continue to present the last defined state of visual media or the timeline state at the end of the active duration (aural media render nothing during the frozen state). The duration of the frozen state depends upon the parent time container as described in Time Container constraints on child durations.
If the fill attribute has the value "hold", the frozen
state lasts until the end of the current simple duration of the
parent time container. The current simple duration refers to one
iteration with a repeating time container, or the (single) instance of the
simple duration for a time container that does not repeat. If the active
duration of the parent time container cuts short the simple duration (e.g.
for repeatCount="2.5"), the frozen state ends when
the active duration of the parent time container ends.
If the fill attribute has the value "freeze", the
frozen state depends upon the parent time container of the current
element:
<par> time container, the frozen state
is the same for both "freeze" and "hold" values of the fill
attribute.
<seq> or <excl> time container,
fill="freeze" specifies that the frozen state lasts until the
next element in the time container begins, or until the end of the current
simple duration of the parent time container (which ever comes
first). See also the section Freezing elements.
It should be noted that for a child of <excl>, the frozen
state does not apply to an element that has been paused, but only to elements
that have ended.
The frozen state may have 0 duration, e.g. if the parent time container ends with the element.
If a timed element has the fill attribute set to
remove (the default), upon reaching the end of its active duration,
the element will transition to the finished state.
In the finished state the timed element does not affect the presentation of the document. The duration of the finished state depends upon the parent time container. The finished state lasts until the end of the current simple duration of the parent time container, or until the element is restarted (whichever comes first).
This state transition can be controlled with the restart attribute.
If the restart attribute value is "always" or "whenNotActive"
the element will transition to the active state in response to an event specified
in begin, a DOM beginElement() method call or a
hyperlink activation. The restart transition effectively resets the state
of the element; the element's simple and active duration must be recomputed
as if it were being started for the first time.
An element may receive an event specified in begin, a
beginElement() DOM call or be the target of a hyperlink activation
while in the active state. In this case, if the value of the
restart attribute is always the element will
re-transition to the active state and restart as described above.
Any other value for the restart attribute will prevent this
transition from occurring.
An element restart can result from an event specified in begin,
a DOM call or a hyperlink activation, subject to the restrictions imposed
by the restart attribute. When in the finished state, an element
may re-transition to the active state if the value of the
restart attribute is always or whenNotActive.
A restart value of never will prevent this
transition from occurring.
In this specification, elements are described as having local "time". In particular, many offsets are computed in the local time of a parent time container. However, simple durations can be repeated, and elements can begin and restart in many ways. As such, there is no direct relationship between the local "time" for an element, and the real world concept of time as reflected on a clock.
When the time manipulation attributes are used to adjust the speed and/or pacing within the simple duration, the semantics can be thought of as changing the pace of time in the given interval. An equivalent model is these attributes simply change the pace at which the presentation progresses through the given interval. The two interpretations are equivalent mathematically, and the significant point is that the notion of "time" as defined for the simple duration and "local time" should not be construed as real world clock time. For the purposes of SMIL Timing and Synchronization, "time" can behave quite differently from real world clock time.
The SMIL timing model assumes the most common model for interval timing. This describes intervals of time (i.e. durations) in which the begin time of the interval is included in the interval, but the end time is excluded from the interval. This is also referred to as "end-point exclusive" timing. This model makes arithmetic for intervals work correctly, and provides sensible models for sequences of intervals.
In the real world, this is equivalent to the way that seconds add up to minutes, and minutes add up to hours. Although a minute is described as 60 seconds, a digital clock never shows more than 59 seconds. Adding one more second to "00:59" does not yield "00:60" but rather "01:00", or 1 minute and 0 seconds. The theoretical end time of 60 seconds that describes a minute interval is excluded from the actual interval.
In the world of media and timelines, the same applies: Let a be a video, a clip of audio, or an animation. Assume "A" begins at 10 and runs until 15 (in any units - it does not matter). If "B" is defined to follow "A", then it begins at 15 (and not at 15 plus some minimum interval). When a runtime actually renders out frames (or samples for audio), and must render the time "15", it should not show both a frame of "A" and a frame of "B", but rather should only show the new element "B". This is the same for audio, or for any interval on a timeline. If the model does not use endpoint-exclusive timing, it will draw overlapping frames, or have overlapping samples of audio, of sequenced animations, etc.
Note that transitions from "A" to "B" also adhere to the interval timing model. They do require that "A" not actually end at 15, and that both elements actually overlap. Nevertheless, the "A" duration is simply extended by the transition duration (e.g. 1 second). This new duration for "A" is also endpoint exclusive - at the end of this new duration, the transition will be complete, and only "B" should be rendered - "A" is no longer needed.
For the time model, several results of this are important: the definition of repeat, and the value sampled during the "frozen" state.
When repeating an element simple duration, the arithmetic follows the end-point exclusive model. Consider the example:
<video dur="4s" repeatCount="4" .../>
At time 0, the simple duration is also at 0, and the first frame of video is presented. This is the inclusive begin of the interval. The simple duration proceeds normally up to 4 seconds. However, the appropriate way to map time on the active duration to time on the simple duration is to use the remainder of division by the simple duration:
simpleTime = REMAINDER( activeTime, d
)
Note: REMAINDER( t, d ) is defined as t -
d*floor(t/d)
Using this, a time of 4 (or 8 or 12) maps to the time of 0 on the simple duration. The endpoint of the simple duration is excluded from (i.e. not actually sampled on) the simple duration.
For most continuous media, this aligns to the internal media model, and so no frames (or audio samples) are ever excluded. However for sampled timeline media (like animation), the distinction is important, and requires a specific semantic for handling the frozen state. This is detailed in the [SMIL-ANIMATION] module.
A significant motivation for SMIL Boston is the desire to integrate declarative, determinate scheduling with interactive, indeterminate scheduling. The goal is to provide a common, consistent model and a simple syntax.
Note that "interactive" content does not refer simply to hypermedia with support for linking between documents, but specifically to content within a presentation (i.e. a document) that is activated by some interactive mechanism (often user-input events, but including local hyperlinking as well).
SMIL Boston describes extensions to SMIL 1.0 to support interactive timing of elements. These extensions allow the author to specify that an element should begin or end in response to an event (such as a user-input event like "click"), or to a hyperlink activation, or to a DOM method call.
The syntax to describe this uses
event-value specifications and the
special argument value "indefinite" for the begin and
end attribute values. Event values describe user interface and
other events. If an element should only begin (or end) with a DOM method
call, the begin and end attributes allow the special
value "indefinite" to indicate this. Setting begin="indefinite"
can also be used when a hyperlink will be used to begin the element. The
element will begin when the hyperlink is actuated (usually by the user clicking
on the anchor). It is not possible to control the active end of an element
using hyperlinks.
SMIL Boston represents an evolution from earlier multimedia runtimes. These were typically either pure, static schedulers or pure event-based systems. Scheduler models present a linear timeline that integrates both discrete and continuous media. Scheduler models tend to be good for storytelling, but have limited support for user-interaction. Event-based systems, on the other hand, model multimedia as a graph of event bindings. Event-based systems provide flexible support for user-interaction, but generally have poor scheduling facilities; they are best applied to highly interactive and experiential multimedia.
The SMIL 1.0 model is primarily a scheduling model, but with some flexibility to support continuous media with unknown duration. User interaction is supported in the form of timed hyperlinking semantics, but there was no support for activating individual elements via interaction.
To integrate interactive content into SMIL timing, the SMIL 1.0 scheduler model is extended to support several new concepts: indeterminate timing and event-activation.
With indeterminate timing, an element has an undefined begin or
active end time. The element still exists within the constraints of
the document, but the begin or active end time is determined by some external
activation. Activation may be event-based (such as by a user-input
event), hyperlink based (with a hyperlink targeted at the element), or DOM
based (by a call to the beginElement() method). From a
scheduling perspective, the time can be thought of as
unresolved.
The event-activation support provides a means of associating an event with the begin or active end time for an element. When the event is raised (e.g. when the user clicks on something), the associated time is resolved to a determinate time. For event-based begin times, the element becomes active (begins to play) at the time that the event is raised. The element plays from the beginning of the media (subject to any explicit clipBegin). For event-based active end times, the element becomes inactive (stops playing) when the associated event is raised.
Note that an event based end will not be activated until the
element has already begun. Any specified end event is ignored
before the element begins.
The constraints imposed on an element by its time container are an important aspect of the event-activation model. In particular, when a time container is itself inactive (e.g. before it begins or after it ends), no events are handled by the children. If the time container is frozen, no events are handled by the children. No event-activation takes place unless the time container of an element is active. For example:
<par begin="10s" dur="5s"> <audio src="song1.au" beginEvent="btn1.click" /> </par>
If the user clicks on the "btn1" element before 10 seconds, or after 15 seconds,
the audio element will not play. In addition, if the audio element
begins but would extend beyond the specified active end of the
<par> container, it is effectively cut off by the active end
of the <par> container.
The semantics of element sensitivity to events are described by the following set of rules:
begin specifies the event, the element begins and any specification
of the event in end is ignored for this event instance.
begin specifies the event, then the behavior depends upon the
value of restart:
restart="always", then the element restarts and any specification
of the event in end is ignored for this event instance.
restart="never" or restart="whenNotActive",
then any begin specification of the event is ignored for this
instance of the event. If end specifies the event, the element
active duration ends.
These rules can be used with the restart attribute to describe "toggle" activation use- cases, as described in the section: Using restart for toggle activation.
Related to event-activation is link-activation. Hyperlinking has defined semantics in SMIL 1.0 to seek a document to a point in time. When combined with indeterminate timing, hyperlinking yields a variant on interactive content. A hyperlink can be targeted at an element that does not have a scheduled begin time. When the link is traversed, the element begins. Note that unlike event activation, the hyperlink activation is not subject to the constraints of the parent time container. The details of when hyperlinks activate an element, and when they seek the document timeline are presented in the next section.
Hyperlinking semantics must be specifically defined within the time model in order to ensure predictable behavior. Earlier hyperlinking semantics, such as those defined by SMIL 1.0 are insufficient because they do not handle indeterminate and interactive timing, nor do they handle author-time restart restrictions. Here we extend SMIL 1.0 semantics for use in presentations using elements with indeterminate timing, interactive timing, and author-time restart restrictions.
A hyperlink may be targeted at a timed element by specifying the value of
the id attribute of an element in the fragment part of the link
locator. Traversing a hyperlink that refers to a timed element will affect
the presentation in one of three ways, depending upon the value of the element's
begin attribute and the current resolution status of
the element's begin time:
begin attribute is equal
to "indefinite", the element will simply be started when a link targeted
at the element is traversed. In this case the semantics of the
restart attribute will govern the behavior of the element if
subsequent hyperlink traversals occur.
begin attribute is not equal
to "indefinite", and the begin time is currently resolved (i.e. for the current
simple duration of the parent time container) will seek the presentation
to that resolved time and continue normally from that point. In this case
the semantics of the restart attribute will not apply to the
activation of the hyperlink.
begin attribute is not equal
to "indefinite", and the begin time is unresolved will not seek the document
timeline and will have no effect upon the referenced element. This should
not be considered an error, and should not affect other language-defined
link behavior.
@@ Need to describe seeking parents as needed to activate the target.
@@ Should we deal with activating targets that are not marked "indefinite"?
@@ Need to add qualifications on a resolved begin that is outside the current simple duration of the parent time container.
@@ Need to describe that seek should prefer the current simple duration, where it makes sense.
If a seek of the presentation time is required, it may be necessary to seek either forward or backward, depending upon the resolved begin time of the element and the presentation current time at the moment of hyperlink traversal. After seeking a document forward, the document should be in the same state as if the user had allowed the presentation to run normally from the current time until reaching the element begin time (but had otherwise not interacted with the document). In particular, seeking the presentation time forward should also cause any other elements that have resolved begin times between the current time and the seeked-to time to begin. These elements may have ended, or may still be active or frozen at the seeked-to time, depending upon their begin times and active durations. Also any elements currently active at the time of hyperlinking should "fast-forward" over the seek interval. These elements may end the active duration, may be still active or may be frozen once the seek is complete, depending upon their active durations. The net effect is that seeking forward to a presentation time puts the document into a state identical to that as if the document presentation time advanced undisturbed to reach the seek time.
If the resolved activation time for an element that is the target of a hyperlink traversal occurs in the past, the presentation time must seek backwards. Seeking backwards will rewind any elements active during the seek interval and will turn off any elements that are resolved to begin at a time after the seeked-to time. Note that resolved begin times (e.g. a begin associated with an event) are not cleared or lost by seeking to an earlier time. Subject to the rules above for hyperlinks that target timed elements, hyperlinking to elements with resolved begin times will function normally, advancing the presentation time forward to the previously resolved time.
These hyperlinking semantics assume that a record is kept of the resolved begin time for all elements, and this record is available to be used for determining the correct presentation time to seek to. Once resolved, begin times are not cleared by hyperlinking. However, they can be overwritten by subsequent resolutions driven by multiple occurrences of an event (i.e. by restarting). For example:
<par begin="0"> <img id="A" begin="10s" .../> <img id="B" begin="A.begin+5s" .../> <img id="C" begin="click" .../> <img id="D" begin="C.begin+5s" .../> ... <a href="#D">Click here!</a> </par>
The begin time of elements "A" and "B" can be immediately resolved to be at 10 and 15 seconds respectively. The begin of elements "C" and "D" are unresolved when the document starts. Therefore activating the hyperlink will have no effect upon the presentation time or upon elements "C" and "D". Now, assume that "C" is clicked at 25 seconds into the presentation. The click on "C" in turn resolves "D" to begin at 30 seconds. From this point on, traversing the hyperlink will cause the presentation time to be seeked to 30 seconds.
If at 60 seconds into the presentation, the user again clicks on "C", "D" will become re-resolved to a presentation time of 65 seconds. Subsequent activation of the hyperlink will result in the seeking the presentation to 65 seconds.
If the time container were defined to repeat, or could restart, then all indeterminate times for children of the time container are cleared (reset to "indefinite") when the parent time container repeats or restarts.
@@This is rough
For a child of a sequence time container, if beginElement()
is called or a hyperlink targeted to the child is traversed, this seeks the
sequence to the beginning of the child. If the seek is forward in time and
the child does not have resolved begin time, the document time must seek
past any scheduled active end on preceding elements, and then activate the
referenced child. In such a seek, if the currently active element does not
have a resolved active end, it should be ended at the current time. If there
are other intervening siblings (between the currently playing element and
the targeted element), the document time must seek past all scheduled times,
and resolve any unresolved times as seek proceeds (time will resolve to
intermediate values of "now" as this process proceeds). As times are resolved,
all associated time dependents get notified as the intervening elements are
activated and deactivated.
Note that the presentation agent need not actually prepare any media for elements that are seeked over, but it does need to propagate the sync behavior to all time dependents so that the effect of the seek is correct.
@@Need to resolve which events get raised. I think we should raise an end event only for the currently playing element, and a begin for the referenced element. Why? Because if an element will never actually play, script need not be notified, and we can preclude a flood of events that get collected during a seek operation.
@@ Compare this discussion to epsilon model.
There are several cases in which times may change as the document is presented. In particular, when an element time is defined relative to an event, the time (i.e. the element begin or active end) is resolved when the event occurs. Another case arises with restart behavior - both the begin and active end time of an element can change when it restarts. Since the begin and active end times of one element can be defined relative to the begin or active end of other elements, any changes to times must be propagated throughout the document.
When an element "foo" has a begin or active end time that specifies a syncbase element (e.g. "bar" as below):
<img id="foo" begin="bar.end" .../>
we say that "foo" is a time-dependent of "bar" - that is, the "foo" begin time depends upon the active end of "bar". Any changes to the active end time of "bar" must be propagated to the begin of "foo". The effect of the changes depends upon the state of "foo" when the change happens. The rest of the section describes the specific rules for propagating changes.
Note that it is possible for the syncbase element "bar" to end again, if it is restarted. When "bar" restarts, the a new end time is calculated and all time dependents are notified of the change. For example:
<img id="foo" begin="0" end="bar.end" .../> <img id="bar" begin="btn.click" dur="5s" .../>
Element "foo" will end when "bar" ends, however "bar" can restart on another click. When "bar" restarts, a new end is calculated, and "bar" is notified. However, as "bar" will not restart, the change is ignored. A variant on this illustrates a case when the time change does propagate through:
<img id="foo" begin="0" end="bar.end+10s" .../> <img id="bar" begin="btn.click" dur="5s" .../>
Element "foo" will end 10 seconds after "bar" ends. If "bar" is restarted within 10 seconds of when it first ended, "foo" will still be active, and the changed end time will propagate through. Using example times, if the user clicks on the "btn" element 8 seconds after the parent time container begins, "bar" begins at 8 seconds and will end at 13 seconds. Element "foo" would then end at 23 seconds. If the users clicks "btn" again 3 seconds after "bar" ends, (i.e. at 16 seconds), the end of "bar" now has the value of 21 seconds. This change propagates to "foo", and "pushes out" the end of "foo" until 31 seconds.
The rule is that once an element has ended its active duration, changes that affect its end time are ignored (within the current simple duration of the parent time container).
When an element restarts (or when an ascendant time container repeats or restarts), all child times are recalculated, and may again become indefinite. For example:
<img id="foo" begin="btn.click" end="mouseout" .../> <img id="bar" begin="btn.click" end="foo.end" .../>
Both elements will start when the "btn" element is first clicked. Element "foo" will end when "mouseout" is raised on the img. At this point, the active duration of "bar" will become defined (resolved), and "bar" will end the active duration. If the user clicks on the target element again, both elements will restart, and "bar" will once again have an indefinite active duration.
Proposed new support in SMIL Boston introduces finer grained control over the runtime synchronization behavior of a document. The syncBehavior attribute allows an author to describe for each element whether it must remain in a hard sync relationship to the parent time container, or whether it can be allowed slip with respect to the time container. Thus, if network congestion delays or interrupts the delivery of media for an element, the syncBehavior attribute controls whether the media element can slip while the rest of the document continues to play, or whether the time container must also wait until the media delivery catches up.
The syncBehavior attribute can also be applied to time containers. This controls the sync relationship of the entire timeline defined by the time container. In this example, the audio and video elements are defined with hard or "locked" sync to maintain lip sync, but the "speech" <par> time container is allowed to slip:
<par>
<animation src="..." />
...
<par id="speech" syncBehavior="canSlip" >
<video src="speech.mpg"
syncBehavior="locked" />
<audio src="speech.au"
syncBehavior="locked" />
</par>
...
</par>
If either the video or audio must pause due to delivery problems, the entire "speech" par will pause, to keep the entire timeline in sync. However, the rest of the document, including the animation element will continue to play normally. Using the syncBehavior attribute on elements and time containers, the author can effectively describe the "scope" of runtime sync behavior, defining some portions of the document to play in hard sync without requiring that the entire document use hard synchronization.
This functionality also applies when an element first begins, and the media must begin to play. If the media is not yet ready (e.g. if an image file has not yet downloaded), the syncBehavior attribute controls whether the time container must wait until the element media is ready, or whether the element begin can slip until the media is downloaded.
The syncBehavior can affect the effective begin and effective end of an element, but the use of the syncBehavior attribute does not introduce any other semantics with respect to duration.
When the syncBehavior attribute is combined with interactive begin timing
for an element, the syncBehavior only applies once the sync relationship
of the element is resolved (e.g. when the specified event is raised). If
at that point the media is not ready and syncBehavior is specified as "locked",
then the parent time container must wait until the media is ready. Once an
element with an interactive begin time has begun playing, the syncBehavior
semantics described above apply as thought the element were defined with
scheduled timing.
Note that the semantics of syncBehavior do not describe or require a particular
approach to maintaining sync; the approach will be implementation dependent.
Possible means of resolving a sync conflict may include:
Additional control is provided over the hard sync model using the syncTolerance attribute. This specifies the amount of slip that can be ignored for an element. Small variance in media playback (e.g. due to hardware inaccuracies) can often be ignored, and allow the overall performance to appear smoother.
@@ Need to make clear that syncBehavior does not inherit, but does default to the inherited value for defaultSyncBehavior. Need some intro the the default*** attributes
@@ Need (somewhere) to specify that once event-based timing is resolved, the element is tied to the parent time container for the purposes of runtime sync management. Similar issue for long sync arcs
@@Should we say the following, or leave it to implementations? There may be a non-zero default value for this attribute, to be determined through testing (a proposed guess is something between 0.1 seconds and 0.5 seconds).
syncBehavior="locked". This allows a locked sync relationship
to ignore a given amount of slew without forcing resynchronization. resolution.
An additional proposed extension allows the author to specify that a particular element should define or control the synchronization for a time container. This is similar to the default behavior of many players that "slave" video and other elements to audio, to accommodate the audio hardware inaccuracies and the sensitivity of listeners to interruptions in the audio playback. The syncMaster attribute allows an author to explicitly define that an element defines the playback "clock" for the time container, and all other elements should be held in sync relative to the syncMaster element.
In practice, linear media often need to be the syncMaster, where non-linear media can more easily be adjusted to maintain hard sync. However, a player cannot always determine which media behaves in a linear fashion and which media behaves in a non-linear fashion. In addition, when there are multiple linear elements active at a given point in time, the player cannot always make the "right" decision to resolve sync conflicts. The syncMaster attribute allows the author to specify the element that has linear media, or that is "most important" and should not be compromised by the syncBehavior of other elements.
The syncMaster attribute interacts with the syncBehavior attribute. An element with syncMaster set to true will define sync for the "scope" of the time container's synchronization behavior. That is, if the syncMaster element's parent time container has syncBehavior="locked", the syncMaster will also define sync for the ancestor time container. The syncMaster will define sync for everything within the closest ancestor time container that is defined with syncBehavior="canSlip".
The syncMaster attribute only applies when an element is active. If more than one element within the syncBehavior scope has the syncMaster attribute set to true, and the elements are both active at any moment in time, the behavior will be implementation dependent.
The default value is false.
The associated property is read-only, and cannot be set by script.
@@ Need to decide whether endSync belongs on media elements (with timed children) or not.
Timing attributes
<!ENTITY % timingAttrs
begin CDATA #IMPLIED
dur CDATA #IMPLIED
end CDATA #IMPLIED
restart (always | never |
whenNotActive) "always"
repeatCount CDATA #IMPLIED
repeatDur CDATA #IMPLIED
fill (remove | freeze | hold) "remove"
>
Runtime sync behavior attributes
<!ENTITY % runtimeSyncBvrAttrs syncBehavior (locked | canSlip) #IMPLIED defaultSyncBehavior (locked | canSlip) "canSlip" syncTolerance CDATA #IMPLIED defaultSyncTolerance CDATA #IMPLIED syncMaster (true | false) "false" >
Time container elements
<!ELEMENT par ???> <!ATTLIST par %timingAttrs %runtimeSyncBvrAttrs id ID #IMPLIED endSync CDATA #IMPLIED >
<!ELEMENT seq ???> <!ATTLIST seq %timingAttrs %runtimeSyncBvrAttrs id ID #IMPLIED >
<!ELEMENT excl ???> <!ATTLIST excl %timingAttrs %runtimeSyncBvrAttrs id ID #IMPLIED endSync CDATA #IMPLIED >
Any XML-based language that integrates SMIL Timing will inherit the basic interfaces defined in DOM [DOM2] (although not all languages may require a DOM implementation). SMIL Timing specifies the interaction of timing functionality and DOM. SMIL Timing also defines constraints upon the basic DOM interfaces, and specific DOM interfaces to support SMIL Timing.
Much of the related SMIL-DOM functionality is proposed in the [SMIL-DOM] section. We may need to go into further detail on the specific semantics of the interfaces - the sections below are placeholders.
Define rules on element and attribute access (inherit from and point to Core DOM docs for this). Define mutation constraints. This is currently covered in the [SMIL-DOM] section.
SMIL event-timing assumes that the host language supports events, and that the events can be bound in a declarative manner. DOM Level 2 Events [DOM2Events] describes functionality to support this.
The specific events supported are defined by the host language. If no events are defined by a host language, event-timing is effectively omitted.
The [SMIL-DOM] section defines the initial set of time-related events that have been proposed.
@@We may support other methods in SMIL Boston, in addition to these.
SMIL Timing supports two methods for controlling the timing of elements:
beginElement() and endElement(). These methods
are used to begin and end the active duration of an element. Authors can
(but are not required to) declare the timing to respond to the DOM using
the following syntax:
<img begin="indefinite" end="indefinite" .../>
@@Need to do more work on rules for beginElement, and interaction with time container constraints. Should beginElement work like a hyperlink activation, or like an event, or like a long sync-arc, or should it just make the element runnable when the parent begins?.
Calling beginElement() causes the element to begin in the same
way that an element with event-based begin timing begins
(@@ This is still under discussion, and may well
change). The effective begin time is the current presentation time
at the time of the DOM method call. Note that beginElement()
is subject to the restart attribute in the same manner that
event-based begin timing is. If an element is specified to disallow restarting
at a given point, beginElement() methods calls must fail. Refer
also to the section Restarting elements.
Calling endElement() causes an element to end the active duration,
just as end does. Depending upon the value of the
fill attribute, the element effect may no longer be applied,
or it may be frozen at the current effect. Refer also to the section
Freezing elements. If an element is not currently
active (i.e. if it has not yet begun or if it is frozen), the
endElement() method will fail.
Interfaces are currently defined in the [SMIL-DOM] section.
The following concepts are the basic terms used to describe the timing model.
A time graph is used to represent the temporal relations of elements in a document with SMIL timing. Nodes of the time graph represent elements in the document. Parent nodes can "contain" children, and children have a single parent. Siblings are elements that have a common parent. The links or "arcs" of the time graph represent synchronization relationships between the nodes of the graph.
Note that this definition is preliminary.
The time model description uses a set of adjectives to describe particular concepts of timing:
An element is considered to have scheduled timing if the element's start time is given relative to the begin or active end of another element. A scheduled element can be inserted directly into the time graph.
Begin and active end times in SMIL Boston can be specified to be relative
to events that are raised in the document playback environment. This
supports declarative, interactive timing. Interactive in this
sense includes user events such as mouse clicks, events raised by media players
like a mediaComplete event, and events raised by the presentation
engine itself such as a pause event.
More information on the supported events and the underlying mechanism is described in the DOM section of this draft [SMIL-DOM].
In scheduled timing, elements are timed relative to other elements. The timebase for an element A is the other element B to which element A is relative. More precisely, it is the begin or active end of the other element. The timebase is not simply a scheduled point in time, but rather a point in the time graph.
Note that this definition is preliminary. The name may also change.
"Sync-arc" is an abbreviation for "synchronization arc". Sync-arcs are used to relate nodes in the time graph, and define the timing relationship between the nodes. A sync-arc relates an element to its timebase. The sync-arc may be defined implicitly by context, explicitly by id-ref or event name, or logically with special syntax.
Note that this definition is preliminary.
A Clock is a particular timeline reference that can be used for synchronization. A common example that uses real-world local time is referred to as wall-clock timing (e.g. specifying 10:30 local time). Other clocks may also be supported by a given presentation environment.
A hyperlink into or within a timed document may cause a seek of the current presentation time or may activate an element (if it is not in violation of any timing model rules).
During playback, an element may be activated automatically by the progression
of time, via a hyperlink, or in response to an event. When an element is
activated, playback of the element begins.
SMIL includes support for declaring media, using element syntax defined in [SMIL-MEDIA]. The media that is described by these elements is described as either discrete or continuous:
Time containers group elements together in time. They define common, simple synchronization relationships among the grouped child elements. In addition, time containers constrain the time that children may be active. Several containers are defined, each with specific semantics and constraints on its children.
SMIL timing and synchronization support ultimately controls a set of content or media elements. The content includes things like video and audio, images and vector graphics, as well as text or HTML content. SMIL documents use the SMIL media elements to reference this content. XML and HTML documents that integrate SMIL Boston functionality may use SMIL media elements and/or content described by the integrated language (e.g. paragraphs in HTML).
All elements - content/media as well as time containers - support timing markup to describe a begin time and a duration, as well as the ability to play repeatedly. There are several ways to define the begin time. The semantics vary somewhat depending upon an element's time container.
The time model defines two concepts of duration for each element - the simple duration and the active duration. These definitions are closely related to the concept of playing something repeatedly.
The constraints of a parent time container may override the duration of its children. In particular, a child element may not play beyond the simple end of the time container.
The terms for these durations can be modified with the Descriptive Terms for Times, to further distinguish aspects of the time graph.
Time manipulations allow the element's time (within the simple duration) to be filtered or modified. For example the speed of time can be varied to make the element play faster or slower than normal. The filtered time affects all descendents of the element. Several time manipulations are proposed for SMIL Boston. Time manipulation is primarily intended to be used with animation [SMIL-ANIMATION] (W3C members only).
Note that any time manipulation that changes the effective play speed of an element's time may conflict with the basic capabilities of some media players. The use of these manipulations is not recommended with linear media players, or with time containers that contain linear media elements, such as streaming video.
There are a number of unresolved issues with this kind of time manipulation, including issues related to event-based timing and negative play speeds, as well as many media-related issues.
Using simple, scheduled timing, a time graph can be described in which all the times have a known, defined sync relationship to the document timeline. We describe this as determinate timing.
When timing is specified relative to events or external clocks, the sync relationship is not initially defined. We describe this as indeterminate timing.
A time is resolved when the sync relationship is defined, and the time can actually be scheduled on the document time graph.
Indeterminate times that are event-based are resolved when the associated event occurs at runtime - this is described more completely in the section Unifying Scheduling and Interactive Timing. Indeterminate times that are defined relative to external clocks are usually resolved when the document playback begins, and the relationship of the document timeline to the external clock reference is defined.
A determinate time may initially be unresolved, e.g. if it is relative to an unknown time such as the end of a streaming MPEG movie (the duration of an MPEG movie is not known until the entire file is downloaded). When the movie finishes, determinate times defined relative to the end of the movie are resolved.
SMIL 1.0 introduced the notion of synchronization behavior, describing user agent behavior as implementing either "hard synchronization" or "soft synchronization". Using hard sync, the entire presentation would be constrained to the strict description of sync relationships in the time graph. Soft sync allowed for a looser (implementation dependent) performance of the document.
While a document is playing, network congestion and other factors will sometimes interfere with normal playback of media. In a SMIL 1.0 hard sync environment, this will affect the behavior of the entire document. In order to provide greater control to authors, SMIL Boston extends the hard and soft sync model to individual elements. This support allows authors to define which elements and time containers must remain in strict or "hard" sync, and which elements and time containers can have a "soft" or slip sync relationship to the parent time container.
This section includes a set of examples that illustrate both the usage of the SMIL syntax, as well as the semantics of specific constructs. This section is informative.
Note: In the examples below, the additional syntax related to layout and other issues specific to individual document types is omitted for simplicity.
All the children of a <par> begin by default when the <par> begins. For example:
<par>
<img id="i1" dur="5s" src="img.jpg" />
<img id="i2" dur="10s" src="img2.jpg" />
<img id="i3" begin="2s" dur="5s" src="img3.jpg"
/>
</par>
Elements "i1" and "i2" both begin immediately when the par begins, which is the default begin time. The active duration of "i1" ends at 5 seconds into the <par>. The active duration of "i2" ends at 10 seconds into the <par>. The last element "i3" begins at 2 seconds since it has an explicit begin offset, and has a duration of 5 seconds which means its active duration ends 7 seconds after the <par> begins.
An image that illustrated the timeline might be useful here.
Each child of a <seq> begins by default when the previous element ends. For example:
<seq>
<img id="i1" begin="0s" dur="5s" src="img1.jpg"
/>
<img id="i2" dur="10s" src="img2.jpg" />
<img id="i3" begin="1s" dur="5s" src="img3.jpg"
/>
</seq>
The element "i1" begins immediately, with the start of the <seq>, and ends 5 seconds later. Note: specifying a begin time of 0 seconds is optional since the default begin offset is always 0 seconds. The second element "i2" begins, by default, 0 seconds after the previous element "i1" ends, which is 5 seconds into the <seq>. Element "i2" ends 10 seconds later, at 15 seconds into the <seq>. The last element, "i3", has a begin offset of 1 second specified, so it begins 1 second after the previous element "i2" ends, and has a duration of 5 seconds, so it ends at 21 seconds into the <seq>.
Insert illustration.
<par> <excl> <par id="p1"> ... </par> <par id="p2"> ... </par> </excl> <a href="p1"><img src="Button1.jpg"/></a> <a href="p2"><img src="Button2.jpg"/></a> </par>
This example models jukebox-like behavior. Clicking on the first image activates the media items of parallel container "p1". If the link on the second image is traversed, "p2" is started (thereby deactivating "p1" if it would still be active).
Shouldn't we say, here, exactly where the elements of the selected par in the excl should begin when a click happens, e.g., if we are 10 seconds into the outer par and we click on button 2, does the MPG video in p2 start 10 seconds into its stream (in-sync), or does it start at its time 0?
Note that the specific syntax for beginEvent argument values is still under discussion.
<par> <excl> <par beginEvent="btn1.click"> ... </par> <par beginEvent="btn2.click"> ... </par> </excl> <img id="btn1" src=... /> <img id="btn2" src=... /> </par>
The same jukebox example, using event-based activation.
In these two examples event-based and anchor-based activation look almost identical, maybe we should come up with examples showing the difference and the relative power of each.
<excl> <ref id="a" begin="0s" ... /> <ref id="b" begin="5s" ... /> </excl>
In the example above, the beginning of "b" deactivates "a" (assuming that a is still active after 5 seconds). Note that this could also be modeled using a sequence with an explicit duration on the children. While the determinate syntax is allowed, this is not expected to be a common use-case scenario.
Issue - should we preclude the use of determinate timing on children of
excl? Other proposals would declare one child (possibly the first)
to begin playing by default. Proposals include an attribute on the
<excl> container that indicate one child to begin playing by default.
For simple media elements (i.e. media elements that are not time containers) that reference discrete media, the implicit duration is defined to be indefinite. This can lead to surprising results, as in this example:
<seq>
<img src="img1.jpg" />
<video src="vid2.mpg" />
<video src="vid3.mpg" />
</seq>
The default syncbase of a sequence is defined to be the effective active end of the previous element in the sequence, unless the active duration is indefinite in which case the default syncbase is the begin of the previous element. In the example, the implicit duration of the image is used to defined the simple and active durations. As a result, the default begin of the second element causes it to begin at the same time as the image. Thus, the image will not show at all! Authors will generally specify an explicit duration for any discrete media elements.
There is an important difference between the semantics of end and dur.
The dur attribute, in conjunction with the begin time, specifies the simple
duration for an element.
This is the duration that is repeated when the element also has a repeat specified. The attribute end on the other hand overrides the active duration of the element. If the element does not have repeat specified, the active duration is the same as the simple duration. However, if the element has repeat specified, then the end will override the repeat, but will not affect the simple duration. For example:
<seq repeat="10" end="stopBtn.click"> <img src="img1.jpg" dur="2s" /> <img src="img2.jpg" dur="2s" /> <img src="img3.jpg" dur="2s" /> </seq>
The sequence will play for 6 seconds on each repeat iteration. It will play through 10 times, unless the user clicks on a "stopBtn" element before 60 seconds have elapsed.
When an implementation supports the SMIL-DOM, it will be possible to make an element begin or end the active duration using script or some other browser extension. When an author wishes to describe an element as interactive in this manner, the following syntax can be used:
<audio src="song1.au" begin="indefinite" />
The element will not begin until the SMIL-DOM beginElement() method is called.