You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
FCP XML supports <title> elements.
AAF probably supports titles too. Someone will need to research this...
Looking at how those formats represent titles, we should be able to model them as a subclass of Marker. Formatting information will likely be hard to normalize across all the formats, but in the spirit of OTIO, we can focus on the content and timing of the titles as that is likely to be the most useful part in practice.
The use cases for subtitles in OTIO seems to relate to swapping in/out localized subtitles, and/or passing subtitles up or downstream in a production pipeline.
Are the positions as coordinates or something like align: leftalign: center? I saw the alignment approach in webvtt I think. If we use that approach we could leave it up to the adapter or the final consumer of the subtitles to decide the actual position, can't we?
Annotations can be drawings, text etc overlaid the video. There's been discussions on how to implement this and that got tied to the position coordinate system.
Yes, those aligns are the ones I was thinking about. It could be up to the adapters, but if we want to have a common ground schema for several subtitle formats it's worth considering to have a way of storing positions.
I don't think waiting for a coordinate system should hold you back from working on this. We can always revisit positioning at a later stage.
Just my two cents (since I looked at this issue my self a few weeks ago :)
Here's what I thought we could start with, based on what I understood from TTML2 and with how I've used captions in Premiere Pro. Please let me know any suggestions/changes/improvements.
I think storing a Style object with each TimedText is wasteful, but decided to have this approach because we don't have id system in otio yet. In TTML2 there's a section to specify all styles and then refer then in each TimedText. With the current approach conversion from otio to say TTML2 won't be so direct because we don't have a single set of styles. One other thing I thought of was to have a list of styles inside Subtitle and then have a string id in each TimedText. But then each TimedText object will have no style meaning independently/outside the Subtitle. Thoughts?
What are Markers used for? Each marker has a markedRange. When we trim an Item or change the sourceRange shouldn't the markedRange of the markers at the start or end of the Item also change? I couldn't find anything that does something like this. Or is this handled in another way? Why I'm asking is if we trim the Subtitle or change its sourceRange, the ranges of TimedTexts at the start or end might change.
Referring to styles is pretty conventional across the board, think CSS. If style is stored per caption, to accommodate the way real systems work, a consumer of a TimedText object would have to perform an initial pass to gather all the styles, and de-duplicate them into a dictionary. It seems to me that an internal referencing scheme for OTIO would be a prerequisite to implementing TimedText in a useful way. As a suggestion, such a referencing scheme could be as simple as saying that there is a concept of named dictionaries on the root object, and the style field in a TimedText refers to an object at e.g. @dict/styles/Japanese_bold. Introducing a root dictionary would open a can of worms of questionable usages, so I think we'd have to be very cautious about unintended consequences. At the same time, style per TimedText seems unworkable IMO.
I'm interested in reading titles and captions out of FCPX (and maybe simple effects like generators), I could try adding them to the adapter but I don't know how they should be represented in OTIO?
@Laurian There's a WIP PR (#805) that adds a representation in OTIO. I was working on an SRT adapter, but there's work left to be done on the OTIO representation to support styles.
Cool, I'll use your TimedText.1 schema to represent things and look into how to do the import/export on the fcp_xml.py adapter for it, as I'm quite familiar with the FCPX format; I can map to that both <title> and <caption> elements, even when <title> points to a generator, the text data is in there and I guess I can "hide" all the other extra bits in the metadata field for now (is there a policy/format on how to pass adapter specific metadata into the timeline?)
@meshula FCPX will have for subtitles placement and styling metadata:
<captionname="People assume that time is a strict progression of cause to effect,"lane="1"offset="15024/300s"duration="8600/2500s"start="3600s"role="iTT?captionFormat=ITT.en-GB">
<textplacement="top">
<text-styleref="ts2">People assume that time is a strict progression of cause to effect,</text-style>
</text>
<text-style-defid="ts2">
<text-stylefont=".AppleSystemUIFont"fontSize="13.01"fontFace="Regular"fontColor="1 0.999974 0.999991 1"backgroundColor="0 0 0 1"/>
</text-style-def>
</caption>
where ref="r6" points to the Apple Motion effect <effect id="r6" name="Continuous" uid=".../Titles.localized/Build In:Out.localized/Continuous.localized/Continuous.moti"/>
Similarly custom titles can have a lot of parameters (here's a BBC News caption one):
<titlename="People assume that time is a strict progression of cause to effect, - 02 Subtitle"lane="1"offset="125200/2500s"ref="r7"duration="13209600/3840000s"start="3600s">
<paramname="Layout Method"key="9999/10201/3000298778/10202/2/314"value="1 (Paragraph)"/>
<paramname="Left Margin"key="9999/10201/3000298778/10202/2/323"value="0"/>
<paramname="Right Margin"key="9999/10201/3000298778/10202/2/324"value="0"/>
<paramname="Top Margin"key="9999/10201/3000298778/10202/2/325"value="0"/>
<paramname="Bottom Margin"key="9999/10201/3000298778/10202/2/326"value="-540"/>
<paramname="Alignment"key="9999/10201/3000298778/10202/2/354/10038/401"value="1 (Center)"/>
<paramname="Line Spacing"key="9999/10201/3000298778/10202/2/354/10038/404"value="-14"/>
<paramname="Alignment"key="9999/10201/3000298778/10202/2/373"value="0 (Left) 2 (Bottom)"/>
<paramname="Source Object"key="9999/10201/3000298778/10202/4/3000449521/201"value="3000449347"/>
<paramname="Scale"key="9999/10201/3000298778/10202/4/3000449521/204"value="-0.0925926"/>
<paramname="Apply Mode"key="9999/10201/3000298778/10202/4/3000450074/200"value="1 (Multiply by source)"/>
<paramname="Source Object"key="9999/10201/3000298778/10202/4/3000450074/201"value="3000450130"/>
<paramname="Scale"key="9999/10201/3000298778/10202/4/3000450074/204"value="10"/>
<paramname="Opacity"key="9999/10201/3000298778/10202/4/3001050732/1000/1044"value="0"/>
<paramname="Speed"key="9999/10201/3000298778/10202/4/3001050732/201/208"value="6 (Custom)"/>
<paramname="Custom Speed"key="9999/10201/3000298778/10202/4/3001050732/201/209">
<keyframeAnimation>
<keyframetime="0s"value="0"/>
<keyframetime="454656/153600s"value="0"/>
</keyframeAnimation>
</param>
<paramname="Range"key="9999/10201/3000298778/10202/4/3001050732/201/229/230"value="6 (Line)"/>
<paramname="End Index"key="9999/10201/3000298778/10202/4/3001050732/201/229/232"value="3"/>
<paramname="Invert"key="9999/10201/3000298778/10202/4/3001050732/201/229/233"value="1"/>
<text>
<text-styleref="ts2">People assume that time is a strict progression of cause to effect,</text-style>
</text>
<text-style-defid="ts2">
<text-stylefont="BBC Reith Sans"fontSize="58"fontFace="Regular"fontColor="1 0.999974 0.999991 1"alignment="center"lineSpacing="-14"/>
</text-style-def>
</title>
again the ref="r7" will point to the actual Apple Motion file <effect id="r7" name="02 Subtitle" uid="~/Titles.localized/BBC News/B Info/02 Subtitle/02 Subtitle.moti" src="file:///Users/laurian/Movies/Motion%20Templates.localized/Titles.localized/BBC%20News/B%20%20Info/02%20Subtitle/02%20Subtitle.moti"/>
So I would try to preserve that in metadata just in case I need it.
Thanks ~ what prompted my question was wondering whether the extra data fell in the category of "extra non-portable stuff that should be preserved" or the category of "falls into the styling category". The examples seem to show styling/layout plus some animation parameters.
Hello Im making a scriptwriting software with dcc plugins for writing scripts for animation-movies.
I wanted to add a shared timelime for the screenplay and the animations ects. so i naturally though of otio.
I'd like to use otio`s format internally in the software aswell but it seems that handeling text is not clearly defined yet?
Is there someting i can do?
@NicTanghe Yes, there's something to do :) We need a concrete proposal.
I've been working on one here as part of a general text layout engine. I've incorporated @Laurian 's examples to illustrate how they might embed in OTIO.
Activity
jminor commentedon Aug 8, 2017
There are several existing formats for subtitles that could be supported via adapters.
WebVTT: https://w3c.github.io/webvtt/
SSA: https://wiki.multimedia.cx/index.php?title=SubStation_Alpha
SRT, etc.
FCP XML supports
<title>elements.AAF probably supports titles too. Someone will need to research this...
Looking at how those formats represent titles, we should be able to model them as a subclass of Marker. Formatting information will likely be hard to normalize across all the formats, but in the spirit of OTIO, we can focus on the content and timing of the titles as that is likely to be the most useful part in practice.
The use cases for subtitles in OTIO seems to relate to swapping in/out localized subtitles, and/or passing subtitles up or downstream in a production pipeline.
jminor commentedon Sep 27, 2017
Here is some info about how AAF encodes video titles: https://www.amwa.tv/downloads/specifications/AS-05_AAF_Effects_protocol_v1.pdf
I'm not sure if this is the same as a subtitle track. This still needs more research.
jminor commentedon Apr 20, 2018
Here are some more links for reference, though the license for some of these may not be compatible.
http://www.nikse.dk/SubtitleEdit/
https://github.com/ubershmekel/pytitle
https://github.com/pyahmed/sub2xml
KarthikRIyer commentedon Sep 17, 2020
I'd like to attempt this. But I'm still trying to understand how different subtitle formats work.
http://www.nikse.dk/SubtitleEdit/ this seems helpful. It has a common representation for subtitles: https://github.com/SubtitleEdit/subtitleedit/blob/master/libse/Subtitle.cs
Will the license be a problem if we refer to this implementation (because we won't be lifting code directly)?
Jameclarke commentedon Sep 17, 2020
Here is a great article referencing how subtitles are used.
https://jonnyelwyn.co.uk/film-and-video-editing/adding-captions-in-premiere-pro-fix-common-problems/
apetrynet commentedon Sep 17, 2020
Some of the subtitle formats include positioning so I think this ties into the coordinate system and annotations as well.
#773 #771
KarthikRIyer commentedon Sep 17, 2020
What are annotations?
Are the positions as coordinates or something like
align: leftalign: center? I saw the alignment approach in webvtt I think. If we use that approach we could leave it up to the adapter or the final consumer of the subtitles to decide the actual position, can't we?apetrynet commentedon Sep 17, 2020
Annotations can be drawings, text etc overlaid the video. There's been discussions on how to implement this and that got tied to the position coordinate system.
Yes, those
aligns are the ones I was thinking about. It could be up to the adapters, but if we want to have a common ground schema for several subtitle formats it's worth considering to have a way of storing positions.I don't think waiting for a coordinate system should hold you back from working on this. We can always revisit positioning at a later stage.
Just my two cents (since I looked at this issue my self a few weeks ago :)
meshula commentedon Sep 17, 2020
@KarthikRIyer Since the license you reference is GPL, one should not reference that code, nor transliterate it.
reinecke commentedon Sep 17, 2020
Hey @KarthikRIyer, check out TTML2 to get a feeling for what can be expressed in timed text: https://www.w3.org/TR/ttml2/
The Netflix Tech blog has some articles about the complexities of timed text (captions and subtitles). Here are a couple of them:
Hopefully that gives a good jumping off point.
KarthikRIyer commentedon Sep 17, 2020
Thanks @reinecke ! I'll go through these links.
KarthikRIyer commentedon Sep 18, 2020
Here's what I thought we could start with, based on what I understood from TTML2 and with how I've used captions in Premiere Pro. Please let me know any suggestions/changes/improvements.
I think storing a
Styleobject with eachTimedTextis wasteful, but decided to have this approach because we don't have id system in otio yet. In TTML2 there's a section to specify all styles and then refer then in each TimedText. With the current approach conversion from otio to say TTML2 won't be so direct because we don't have a single set of styles. One other thing I thought of was to have a list of styles insideSubtitleand then have a string id in eachTimedText. But then eachTimedTextobject will have no style meaning independently/outside the Subtitle. Thoughts?What are
Markersused for? Each marker has a markedRange. When we trim anItemor change the sourceRange shouldn't the markedRange of the markers at the start or end of theItemalso change? I couldn't find anything that does something like this. Or is this handled in another way? Why I'm asking is if we trim theSubtitleor change its sourceRange, the ranges ofTimedTextsat the start or end might change.meshula commentedon Mar 26, 2021
Referring to styles is pretty conventional across the board, think CSS. If style is stored per caption, to accommodate the way real systems work, a consumer of a TimedText object would have to perform an initial pass to gather all the styles, and de-duplicate them into a dictionary. It seems to me that an internal referencing scheme for OTIO would be a prerequisite to implementing TimedText in a useful way. As a suggestion, such a referencing scheme could be as simple as saying that there is a concept of named dictionaries on the root object, and the style field in a TimedText refers to an object at e.g.
@dict/styles/Japanese_bold. Introducing a root dictionary would open a can of worms of questionable usages, so I think we'd have to be very cautious about unintended consequences. At the same time, style per TimedText seems unworkable IMO.5 remaining items
Laurian commentedon Jul 22, 2021
I'm interested in reading titles and captions out of FCPX (and maybe simple effects like generators), I could try adding them to the adapter but I don't know how they should be represented in OTIO?
KarthikRIyer commentedon Jul 22, 2021
@Laurian There's a WIP PR (#805) that adds a representation in OTIO. I was working on an SRT adapter, but there's work left to be done on the OTIO representation to support styles.
Laurian commentedon Jul 22, 2021
Cool, I'll use your
TimedText.1schema to represent things and look into how to do the import/export on thefcp_xml.pyadapter for it, as I'm quite familiar with the FCPX format; I can map to that both<title>and<caption>elements, even when<title>points to a generator, the text data is in there and I guess I can "hide" all the other extra bits in the metadata field for now (is there a policy/format on how to pass adapter specific metadata into the timeline?)meshula commentedon Jul 23, 2021
I'm curious what the other metadata from FCP contains?
jminor commentedon Jul 23, 2021
@Laurian yes, the general guidance is that an adapter should translate to/from the OTIO schema (in this case @KarthikRIyer 's proposed TimedText schema) and then to put anything else interesting into metadata. Specifically nested into a sub-dictionary within metadata that is clearly labelled. This makes that metadata visible and invites discussion about what else could/should be promoted into the official schema.
You can see more guidance here: https://opentimelineio.readthedocs.io/en/latest/tutorials/otio-file-format-specification.html?highlight=metadata#metadata
and here: https://opentimelineio.readthedocs.io/en/latest/tutorials/write-an-adapter.html?highlight=metadata#metadata
meshula commentedon Jul 25, 2021
This project
https://github.com/naomiaro/waveform-playlist
led me to this -
https://github.com/readbeyond/aeneas
aeneas looks like a gold mine of reference material.
Laurian commentedon Jul 29, 2021
@meshula FCPX will have for subtitles placement and styling metadata:
But other titles you can add can be similar
where
ref="r6"points to the Apple Motion effect<effect id="r6" name="Continuous" uid=".../Titles.localized/Build In:Out.localized/Continuous.localized/Continuous.moti"/>Similarly custom titles can have a lot of parameters (here's a BBC News caption one):
again the
ref="r7"will point to the actual Apple Motion file<effect id="r7" name="02 Subtitle" uid="~/Titles.localized/BBC News/B Info/02 Subtitle/02 Subtitle.moti" src="file:///Users/laurian/Movies/Motion%20Templates.localized/Titles.localized/BBC%20News/B%20%20Info/02%20Subtitle/02%20Subtitle.moti"/>So I would try to preserve that in metadata just in case I need it.
meshula commentedon Jul 29, 2021
Thanks ~ what prompted my question was wondering whether the extra data fell in the category of "extra non-portable stuff that should be preserved" or the category of "falls into the styling category". The examples seem to show styling/layout plus some animation parameters.
NicTanghe commentedon Jul 11, 2026
Hello Im making a scriptwriting software with dcc plugins for writing scripts for animation-movies.
I wanted to add a shared timelime for the screenplay and the animations ects. so i naturally though of otio.
I'd like to use otio`s format internally in the software aswell but it seems that handeling text is not clearly defined yet?
Is there someting i can do?
meshula commentedon Aug 7, 2026
@NicTanghe Yes, there's something to do :) We need a concrete proposal.
I've been working on one here as part of a general text layout engine. I've incorporated @Laurian 's examples to illustrate how they might embed in OTIO.
https://github.com/meshula/LabLayout/blob/dev/docs/otio-caption-proposal.md