Merge branch 'master' of https://github.com/iluvcapra/pycmx

Experiment

README.md

@@ -1,4 +1,5 @@
-[![Build Status](https://travis-ci.com/iluvcapra/pycmx.svg?branch=master)](https://travis-ci.com/iluvcapra/pycmx)
+[![Build Status](https://travis-ci.com/iluvcapra/pycmx.svg?branch=master)](https://travis-ci.com/iluvcapra/pycmx) [![Documentation Status](https://readthedocs.org/projects/pycmx/badge/?version=latest)](https://pycmx.readthedocs.io/en/latest/?badge=latest)
+
 
 # pycmx
 
@@ -6,7 +7,7 @@ The `pycmx` package provides a basic interface for parsing a CMX 3600 EDL and it
 
 ## Features
 
-* The major variations of the CMX3600, the standard, "File32" and "File128" 
+* The major variations of the CMX 3600: the standard, "File32" and "File128" 
   formats are automatically detected and properly read.
 * Preserves relationship between events and individual edits/clips.
 * Remark or comment fields with common recognized forms are read and 
@@ -15,31 +16,49 @@ The `pycmx` package provides a basic interface for parsing a CMX 3600 EDL and it
 * Does not parse or validate timecodes, does not enforce framerates, does not
   parameterize timecode or framerates in any way. This makes the parser more
   tolerant of EDLs with mixed rates.
+* Unrecognized lines are accessible on the `EditList` and `Event` classes
+  along with the line numbers, to help the client diagnose problems with a
+  list and give the client the ability to extend the package with their own
+  parsing code.
 
 ## Usage
 
+### Opening and Parsing EDL Files
 ```
 >>> import pycmx
->>> edl = pycmx.parse_cmx3600("tests/edls/TEST.edl")
+>>> with open("tests/edls/TEST.edl") as f
+... 	edl = pycmx.parse_cmx3600(f)
+...
 >>> edl.title
 'DC7 R1_v8.2'
+```
+
+### Reading Events and Edits
+
+`EditList.events` is a generator...
+
+```
 >>> events = list( edl.events )  
-              # the event list is a generator
 >>> len(events)
 120
 >>> events[43].number 
 '044'
+```
+
+...and events contain 1...n edits.
+
+```
 >>> events[43].edits[0].source_in 
 '00:00:00:00'
 >>> events[43].edits[0].transition.cut
 True
 >>> events[43].edits[0].record_out
 '01:10:21:10'
+```
 
-              # events contain multiple  
+### Acessing Transitions and Enabled Channels
-              # edits to preserve A/B dissolves 
+
-              # and key backgrounds
+```           
-              
 >>> events[41].edits[0].transition.dissolve
 False
 >>> events[41].edits[1].transition.dissolve
@@ -61,14 +80,34 @@ False
 Audio channel 7 is present
 >>> events[2].edits[0].channels.video
 False
-
 ```
 
+## How is this different from `python-edl`?
+
+There are two important differences between `import edl` and `import pycmx` 
+and motivated my development of this module.
+
+1. The `pycmx` parser doesn't take timecode or framerates into account, 
+   and strictly treats timecodes like opaque values. As far as `pycmx` is 
+   concerend, they're just strings. This was done because in my experience, 
+   the frame rate of an EDL is often difficult to precisely determine and 
+   often the frame rate of various sources is different from the frame rate 
+   of the target track.
+   
+   In any event, timecodes in an EDL are a kind of *address* and are not
+   exactly scalar, they're meant to point to a particular block of video or 
+   audio data on a medium and presuming that they refer to a real time, or 
+   duration, or are convertible, etc. isn't always safe.
+
+2. The `pycmx` parser reads event numbers and keeps track of which EDL rows
+   are meant to happen "at the same time," with two decks. This makes it 
+   easier to reconstruct transition A/B clips, and read clip names from
+   such events appropriately.
+ 
 ## Should I Use This?
 
-At this time, this is (at best) alpha software and the interface will be 
+At this time, this is (at best) beta software. I feel like the interface is 
-changing often. It may be fun to experiment with but it is not suitable
+about where where I'd like it to be but more testing is required.
-at this time for production code.
 
 Contributions are welcome and will make this module production-ready all the
 faster! Please reach out or file a ticket! 

bin/edl2scenelist.py

@@ -0,0 +1,102 @@
+import pycmx
+import re
+import argparse
+import sys
+
+import logging
+
+
+FORMAT = '%(asctime)-15s %(message)s'
+logging.basicConfig(format=FORMAT)
+
+
+log = logging.getLogger(__name__)
+
+def all_video_edits(edl):
+    for event in edl.events:
+        for edit in event.edits:
+            if edit.channels.video:
+                yield edit
+
+
+def get_scene_name(edit, pattern):
+    scene_extractor = re.compile(pattern, re.I)
+    if edit.clip_name is None:
+        return None
+    else:
+        match_data = re.match(scene_extractor, edit.clip_name)
+        if match_data:
+            return match_data[1]
+        else:
+            return edit.clip_name
+
+def output_cmx(outfile, out_list):
+    outfile.write("TITLE:  SCENE LIST\r\n")
+    outfile.write("FCM: NON-DROP FRAME\r\n")
+
+    for o in out_list:
+        line = "%03i  AX       V     C        00:00:00:00 00:00:00:00 %s %s\r\n" % (0, o['start'],o['end'])
+        outfile.write(line)
+        outfile.write("* FROM CLIP NAME: %s\r\n" % (o['scene']) )
+
+
+def output_cols(outfile, out_list):
+    for o in out_list:
+        outfile.write("%-12s\t%-12s\t%s\n" % (o['start'], o['end'], o['scene'] ))
+
+
+def scene_list(infile, outfile, out_format, pattern):
+    
+    edl = pycmx.parse_cmx3600(infile)
+
+    current_scene_name = None
+    
+    grouped_edits = [ ]
+
+    for edit in all_video_edits(edl):
+        this_scene_name = get_scene_name(edit, pattern)
+        if this_scene_name is not None:
+            if current_scene_name != this_scene_name:
+                grouped_edits.append([ ])
+                current_scene_name = this_scene_name
+
+            grouped_edits[-1].append(edit)
+    
+    out_list = [ ]
+    for group in grouped_edits:
+        out_list.append({ 
+            'start': group[0].record_in, 
+            'end': group[-1].record_out,
+            'scene': get_scene_name(group[0], pattern ) }
+            )
+    
+    if out_format == 'cmx':
+        output_cmx(outfile, out_list)
+    if out_format == 'cols':
+        output_cols(outfile, out_list)
+    else:
+        log.warning(f"Format {out_format} unrecognized. Will use cmx.\n")
+        output_cmx(outfile, out_list)
+
+
+def scene_list_cli():
+    parser = argparse.ArgumentParser(description=
+            'Read video events from an input CMX EDL and output events merged into scenes.')
+    parser.add_argument('-o','--outfile', default=sys.stdout, type=argparse.FileType('w'), 
+            help='Output file. Default is stdout.')
+    parser.add_argument('-f','--format', default='cmx', type=str, 
+            help='Output format. Options are cols and cmx, cmx is the default.')
+    parser.add_argument('-p','--pattern', default='V?([A-Z]*[0-9]+)',
+            help='RE pattern for extracting scene name from clip name. The default is "V?([A-Z]*[0-9]+)". ' + \
+                    'This pattern will be matched case-insensitively.')
+    parser.add_argument('input_edl', default=sys.stdin, type=argparse.FileType('r'), nargs='?', 
+            help='Input file. Default is stdin.')
+    args = parser.parse_args()
+    
+    infile = args.input_edl
+
+    scene_list(infile=infile, outfile=args.outfile , out_format=args.format, pattern=args.pattern)
+
+
+if __name__ == '__main__':
+   scene_list_cli()

docs/.buildinfo

@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: 9f647f525db0c82aadec132928a40ec5
+tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/.gitignore

@@ -0,0 +1 @@
+build/*

docs/Makefile

@@ -0,0 +1,19 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.http://sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
+
+:end
+popd

docs/source/conf.py

@@ -0,0 +1,186 @@
+# -*- coding: utf-8 -*-
+#
+# Configuration file for the Sphinx documentation builder.
+#
+# This file does only contain a selection of the most common options. For a
+# full list see the documentation:
+# http://www.sphinx-doc.org/en/master/config
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+sys.path.insert(0, os.path.abspath('../../pycmx'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = u'pycmx'
+copyright = u'2018, Jamie Hardt'
+author = u'Jamie Hardt'
+
+# The short X.Y version
+version = u''
+# The full version, including alpha/beta/rc tags
+release = u''
+
+
+# -- General configuration ---------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#
+# needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.todo',
+    'sphinx.ext.coverage',
+    'sphinx.ext.viewcode',
+    'sphinx.ext.githubpages',
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The master toctree document.
+master_doc = 'index'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = []
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = None
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'sphinx_rtd_theme'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#
+html_theme_options = {'collapse_navigation': False}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Custom sidebar templates, must be a dictionary that maps document names
+# to template names.
+#
+# The default sidebars (for documents that don't match any pattern) are
+# defined by theme itself.  Builtin themes are using these templates by
+# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
+# 'searchbox.html']``.
+#
+# html_sidebars = {}
+
+
+# -- Options for HTMLHelp output ---------------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'pycmxdoc'
+
+
+# -- Options for LaTeX output ------------------------------------------------
+
+latex_elements = {
+    # The paper size ('letterpaper' or 'a4paper').
+    #
+    # 'papersize': 'letterpaper',
+
+    # The font size ('10pt', '11pt' or '12pt').
+    #
+    # 'pointsize': '10pt',
+
+    # Additional stuff for the LaTeX preamble.
+    #
+    # 'preamble': '',
+
+    # Latex figure (float) alignment
+    #
+    # 'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+    (master_doc, 'pycmx.tex', u'pycmx Documentation',
+     u'Jamie Hardt', 'manual'),
+]
+
+
+# -- Options for manual page output ------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    (master_doc, 'pycmx', u'pycmx Documentation',
+     [author], 1)
+]
+
+
+# -- Options for Texinfo output ----------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (master_doc, 'pycmx', u'pycmx Documentation',
+     author, 'pycmx', 'One line description of project.',
+     'Miscellaneous'),
+]
+
+
+# -- Options for Epub output -------------------------------------------------
+
+# Bibliographic Dublin Core info.
+epub_title = project
+
+# The unique identifier of the text. This can be a ISBN number
+# or the project homepage.
+#
+# epub_identifier = ''
+
+# A unique identification for the text.
+#
+# epub_uid = ''
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files = ['search.html']
+
+
+# -- Extension configuration -------------------------------------------------
+
+# -- Options for todo extension ----------------------------------------------
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = True

docs/source/index.rst

@@ -0,0 +1,20 @@
+.. pycmx documentation master file, created by
+   sphinx-quickstart on Wed Dec 26 21:51:43 2018.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to pycmx's documentation!
+=================================
+
+.. toctree::
+   :maxdepth: 5
+   :caption: API Reference:
+   
+   pycmx
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

docs/source/modules.rst

@@ -0,0 +1,7 @@
+pycmx
+=====
+
+.. toctree::
+   :maxdepth: 4
+
+   pycmx

docs/source/pycmx.rst

@@ -0,0 +1,46 @@
+pycmx package
+=============
+
+Submodules
+----------
+
+pycmx.channel\_map module
+-------------------------
+
+.. automodule:: pycmx.channel_map
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+pycmx.parse\_cmx\_events module
+-------------------------------
+
+.. automodule:: pycmx.parse_cmx_events
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+pycmx.parse\_cmx\_statements module
+-----------------------------------
+
+.. automodule:: pycmx.parse_cmx_statements
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+pycmx.util module
+-----------------
+
+.. automodule:: pycmx.util
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+
+Module contents
+---------------
+
+.. automodule:: pycmx
+    :members:
+    :undoc-members:
+    :show-inheritance:

pycmx/__init__.py

@@ -1,6 +1,16 @@
-# pycmx init
+# -*- coding: utf-8 -*-
+"""
+pycmx is a module for parsing CMX 3600-style EDLs. For more information and 
+examples see README.md
+
+This module (c) 2018 Jamie Hardt. For more information on your rights to 
+copy and reuse this software, refer to the LICENSE file included with the 
+distribution.
+"""
+
+__version__ = '0.7'
+__author__  = 'Jamie Hardt'
 
 from .parse_cmx_events import parse_cmx3600, Transition, Event, Edit
 from . import parse_cmx_events
 
-__version__ = '0.6'

pycmx/channel_map.py

@@ -9,7 +9,7 @@ class ChannelMap:
     Represents a set of all the channels to which an event applies.
     """
 
-    chan_map = {     "V" :    (True,   False,   False),
+    _chan_map = {     "V" :    (True,   False,   False),
                 "A" :    (False,  True,    False),
                 "A2" :   (False,  False,   True),
                 "AA" :   (False,  True,    True),
@@ -35,6 +35,7 @@ class ChannelMap:
 
     @property
     def a1(self):
+        """True if A1 is included."""
         return self.get_audio_channel(1)
 
     @a1.setter
@@ -43,6 +44,7 @@ class ChannelMap:
 
     @property
     def a2(self):
+        """True if A2 is included."""
         return self.get_audio_channel(2)
 
     @a2.setter
@@ -51,6 +53,7 @@ class ChannelMap:
 
     @property
     def a3(self):
+        """True if A3 is included."""
         return self.get_audio_channel(3)
 
     @a3.setter
@@ -59,6 +62,7 @@ class ChannelMap:
     
     @property
     def a4(self):
+        """True if A4 is included."""
         return self.get_audio_channel(4)
 
     @a4.setter
@@ -66,18 +70,20 @@ class ChannelMap:
         self.set_audio_channel(4,val)
 
     def get_audio_channel(self,chan_num):
+        """True if chan_num is included."""
         return (chan_num in self._audio_channel_set)
 
     def set_audio_channel(self,chan_num,enabled):
+        """If enabled is true, chan_num will be included."""
         if enabled:
             self._audio_channel_set.add(chan_num)
         elif self.get_audio_channel(chan_num):
             self._audio_channel_set.remove(chan_num)
     
-    def append_event(self, event_str):
+    def _append_event(self, event_str):
         alt_channel_re = compile('^A(\d+)')
-        if event_str in self.chan_map:
+        if event_str in self._chan_map:
-            channels = self.chan_map[event_str]
+            channels = self._chan_map[event_str]
             self.v = channels[0]
             self.a1 = channels[1]
             self.a2 = channels[2]
@@ -86,7 +92,7 @@ class ChannelMap:
             if matchresult:
                 self.set_audio_channel(int( matchresult.group(1)), True )
 
-    def append_sxt(self, audio_ext):
+    def _append_sxt(self, audio_ext):
         self.a3 = ext.audio3
         self.a4 = ext.audio4
 

pycmx/parse_cmx_events.py

@@ -2,27 +2,55 @@
 # (c) 2018 Jamie Hardt
 
 from .parse_cmx_statements import (parse_cmx3600_statements, 
-        StmtEvent, StmtFCM, StmtTitle, StmtClipName, StmtSourceFile, StmtAudioExt)
+        StmtEvent, StmtFCM, StmtTitle, StmtClipName, StmtSourceFile, StmtAudioExt, StmtUnrecognized)
 
 from .channel_map import ChannelMap
 
 from collections import namedtuple
 
-def parse_cmx3600(path):
+def parse_cmx3600(f):
-    statements = parse_cmx3600_statements(path)
+    """
+    Parse a CMX 3600 EDL.
+
+    Args:
+        f : a file-like object, anything that's readlines-able.
+
+    Returns:
+        An :obj:`EditList`.
+    """
+    statements = parse_cmx3600_statements(f)
     return EditList(statements)
     
     
 class EditList:
+    """
+    Represents an entire edit decision list as returned by `parse_cmx3600()`.
+    
+    """
     def __init__(self, statements):
         self.title_statement = statements[0]
         self.event_statements = statements[1:]
 
     @property
     def title(self):
+        """
+        The title of this edit list, as attensted by the 'TITLE:' statement on 
+        the first line.
+        """
         'The title of the edit list'
         return self.title_statement.title
 
+    
+    @property
+    def unrecognized_statements(self):
+        """
+        A generator for all the unrecognized statements in the list.
+        """
+        for s in self.event_statements:
+            if type(s) is StmtUnrecognized:
+                yield s
+        
+
     @property
     def events(self):
         'A generator for all the events in the edit list'
@@ -51,52 +79,99 @@ class EditList:
 
 
 class Edit:
-    def __init__(self, edit_statement, audio_ext_statement, clip_name_statement, source_file_statement):
+    def __init__(self, edit_statement, audio_ext_statement, clip_name_statement, source_file_statement, other_statements = []):
         self.edit_statement = edit_statement
         self.audio_ext = audio_ext_statement
         self.clip_name_statement = clip_name_statement
         self.source_file_statement = source_file_statement
+        self.other_statements = other_statements
+
+    @property
+    def line_number(self):
+        """
+        Get the line number for the "standard form" statement associated with
+        this edit. Line numbers a zero-indexed, such that the 
+        "TITLE:" record is line zero.
+        """
+        return self.edit_statement.line_number
 
     @property
     def channels(self):
+        """
+        Get the :obj:`ChannelMap` object associated with this Edit.
+        """
         cm = ChannelMap()
-        cm.append_event(self.edit_statement.channels)
+        cm._append_event(self.edit_statement.channels)
         if self.audio_ext != None:
-            cm.append_ext(self.audio_ext)
+            cm._append_ext(self.audio_ext)
         return cm
 
     @property
     def transition(self):
+        """
+        Get the :obj:`Transition` object associated with this edit.
+        """
         return Transition(self.edit_statement.trans, self.edit_statement.trans_op)
    
     @property
     def source_in(self):
+        """
+        Get the source in timecode.
+        """
         return self.edit_statement.source_in
 
     @property
     def source_out(self):
+        """
+        Get the source out timecode.
+        """
+
         return self.edit_statement.source_out
 
     @property
     def record_in(self):
+        """
+        Get the record in timecode.
+        """
+
         return self.edit_statement.record_in
 
     @property
     def record_out(self):
+        """
+        Get the record out timecode.
+        """
+
         return self.edit_statement.record_out
 
     @property
     def source(self):
+        """
+        Get the source column. This is the 8, 32 or 128-character string on the
+        event record line, this usually references the tape name of the source.
+        """
         return self.edit_statement.source
 
 
     @property
     def source_file(self):
-        return self.source_file_statement.filename
+        """
+        Get the source file, as attested by a "* SOURCE FILE" remark on the
+        EDL. This will return None if the information is not present.
+        """
+        if self.source_file_statement is None:
+            return None
+        else:
+            return self.source_file_statement.filename
 
 
     @property
     def clip_name(self):
+        """
+        Get the clip name, as attested by a "* FROM CLIP NAME" or "* TO CLIP 
+        NAME" remark on the EDL. This will return None if the information is
+        not present.
+        """
         if self.clip_name_statement != None:
             return self.clip_name_statement.name
         else:
@@ -105,15 +180,25 @@ class Edit:
 
 
 class Event:
+    """
+    Represents a collection of :obj:`Edit`s, all with the same event number.
+    """
+
     def __init__(self, statements):
         self.statements = statements
     
     @property
     def number(self):
-        return self._edit_statements()[0].event
+        """Return the event number."""
+        return int(self._edit_statements()[0].event)
 
     @property
     def edits(self):
+        """
+        Returns the edits. Most events will have a single edit, a single event
+        will have multiple edits when a dissolve, wipe or key transition needs
+        to be performed.
+        """
         edits_audio = list( self._statements_with_audio_ext() )
         clip_names  = self._clip_name_statements()
         source_files= self._source_file_statements()
@@ -146,7 +231,14 @@ class Event:
 
         return [ Edit(e1[0],e1[1],n1,s1) for (e1,n1,s1) in zip(*the_zip) ]
             
-                
+    @property
+    def unrecognized_statements(self):
+        """
+        A generator for all the unrecognized statements in the event.
+        """
+        for s in self.statements:
+            if type(s) is StmtUnrecognized:
+                yield s            
 
     def _edit_statements(self):
         return [s for s in self.statements if type(s) is StmtEvent]
@@ -170,12 +262,18 @@ class Transition:
     """Represents a CMX transition, a wipe, dissolve or cut."""
 
     Cut = "C"
+
     Dissolve = "D"
+
     Wipe = "W"
+
     KeyBackground = "KB"
+
     Key = "K"
+
     KeyOut = "KO"
 
+
     def __init__(self, transition, operand):
         self.transition = transition
         self.operand = operand
@@ -184,6 +282,9 @@ class Transition:
 
     @property
     def kind(self):
+        """
+        Return the kind of transition: Cut, Wipe, etc
+        """
         if self.cut:
             return Transition.Cut
         elif self.dissolve:
@@ -216,7 +317,7 @@ class Transition:
 
     @property
     def effect_duration(self):
-        """"`The duration of this transition, in frames of the record target.
+        """The duration of this transition, in frames of the record target.
         
         In the event of a key event, this is the duration of the fade in.
         """

pycmx/parse_cmx_statements.py

@@ -23,14 +23,16 @@ StmtMotionMemory = namedtuple("MotionMemory",["source","fps"]) # FIXME needs mor
 StmtUnrecognized = namedtuple("Unrecognized",["content","line_number"])
 
 
-def parse_cmx3600_statements(path):
+def parse_cmx3600_statements(file):
-    with open(path,'r') as file:
+    """
-        lines = file.readlines()
+    Return a list of every statement in the file argument.
-        line_numbers = count() 
+    """
-        return [parse_cmx3600_line(line.strip(), line_number) \
+    lines = file.readlines()
-                for (line, line_number) in zip(lines,line_numbers)]
+    line_numbers = count() 
+    return [_parse_cmx3600_line(line.strip(), line_number) \
+            for (line, line_number) in zip(lines,line_numbers)]
     
-def edl_column_widths(event_field_length, source_field_length):
+def _edl_column_widths(event_field_length, source_field_length):
     return [event_field_length,2, source_field_length,1,
                             4,2, # chans
                             4,1, # trans
@@ -40,63 +42,63 @@ def edl_column_widths(event_field_length, source_field_length):
                             11,1,
                             11]
 
-def edl_m2_column_widths():
+def _edl_m2_column_widths():
     return [2, # "M2"
             3,3, #
             8,8,1,4,2,1,4,13,3,1,1]
 
 
-def parse_cmx3600_line(line, line_number):
+def _parse_cmx3600_line(line, line_number):
     long_event_num_p  = re.compile("^[0-9]{6} ")
     short_event_num_p = re.compile("^[0-9]{3} ")
     
     if isinstance(line,str):
         if line.startswith("TITLE:"):
-            return parse_title(line,line_number)
+            return _parse_title(line,line_number)
         elif line.startswith("FCM:"):
-            return parse_fcm(line, line_number)
+            return _parse_fcm(line, line_number)
         elif long_event_num_p.match(line) != None:
-            length_file_128 = sum(edl_column_widths(6,128))
+            length_file_128 = sum(_edl_column_widths(6,128))
             if len(line) < length_file_128:
-                return parse_long_standard_form(line, 32, line_number)
+                return _parse_long_standard_form(line, 32, line_number)
             else:
-                return parse_long_standard_form(line, 128, line_number)
+                return _parse_long_standard_form(line, 128, line_number)
         elif short_event_num_p.match(line) != None:
-            return parse_standard_form(line, line_number)
+            return _parse_standard_form(line, line_number)
         elif line.startswith("AUD"):
-            return parse_extended_audio_channels(line,line_number)
+            return _parse_extended_audio_channels(line,line_number)
         elif line.startswith("*"):
-            return parse_remark( line[1:].strip(), line_number)
+            return _parse_remark( line[1:].strip(), line_number)
         elif line.startswith(">>>"):
-            return parse_trailer_statement(line, line_number)
+            return _parse_trailer_statement(line, line_number)
         elif line.startswith("EFFECTS NAME IS"):
-            return parse_effects_name(line, line_number)
+            return _parse_effects_name(line, line_number)
         elif line.startswith("SPLIT:"):
-            return parse_split(line, line_number)
+            return _parse_split(line, line_number)
         elif line.startswith("M2"):
-            return parse_motion_memory(line, line_number)
+            return _parse_motion_memory(line, line_number)
         else:
-            return parse_unrecognized(line, line_number)
+            return _parse_unrecognized(line, line_number)
 
     
-def parse_title(line, line_num):
+def _parse_title(line, line_num):
     title = line[6:].strip()
     return StmtTitle(title=title,line_number=line_num)
 
-def parse_fcm(line, line_num):
+def _parse_fcm(line, line_num):
     val = line[4:].strip()
     if val == "DROP FRAME":
         return StmtFCM(drop= True, line_number=line_num)
     else:
         return StmtFCM(drop= False, line_number=line_num)
 
-def parse_long_standard_form(line,source_field_length, line_number):
+def _parse_long_standard_form(line,source_field_length, line_number):
-    return parse_columns_for_standard_form(line, 6, source_field_length, line_number)
+    return _parse_columns_for_standard_form(line, 6, source_field_length, line_number)
     
-def parse_standard_form(line, line_number):
+def _parse_standard_form(line, line_number):
-    return parse_columns_for_standard_form(line, 3, 8, line_number)
+    return _parse_columns_for_standard_form(line, 3, 8, line_number)
     
-def parse_extended_audio_channels(line, line_number):
+def _parse_extended_audio_channels(line, line_number):
     content = line.strip()
     if content == "AUD   3":
         return StmtAudioExt(audio3=True, audio4=False, line_number=line_number)
@@ -107,7 +109,7 @@ def parse_extended_audio_channels(line, line_number):
     else:
         return StmtUnrecognized(content=line, line_number=line_number)
     
-def parse_remark(line, line_number):
+def _parse_remark(line, line_number):
     if line.startswith("FROM CLIP NAME:"):
         return StmtClipName(name=line[15:].strip() , affect="from", line_number=line_number)
     elif line.startswith("TO CLIP NAME:"):
@@ -117,11 +119,11 @@ def parse_remark(line, line_number):
     else:
         return StmtRemark(text=line, line_number=line_number)
 
-def parse_effects_name(line, line_number):
+def _parse_effects_name(line, line_number):
     name = line[16:].strip()
     return StmtEffectsName(name=name, line_number=line_number)
 
-def parse_split(line, line_number):
+def _parse_split(line, line_number):
     split_type = line[10:21]
     is_video = False
     if split_type.startswith("VIDEO"):
@@ -131,15 +133,15 @@ def parse_split(line, line_number):
     return StmtSplitEdit(video=is_video, magnitude=split_mag, line_number=line_number)
 
 
-def parse_motion_memory(line, line_number):
+def _parse_motion_memory(line, line_number):
     return StmtMotionMemory(source = "", fps="")
 
 
-def parse_unrecognized(line, line_number):
+def _parse_unrecognized(line, line_number):
     return StmtUnrecognized(content=line, line_number=line_number)
 
-def parse_columns_for_standard_form(line, event_field_length, source_field_length, line_number):
+def _parse_columns_for_standard_form(line, event_field_length, source_field_length, line_number):
-    col_widths = edl_column_widths(event_field_length, source_field_length)
+    col_widths = _edl_column_widths(event_field_length, source_field_length)
     
     if sum(col_widths) > len(line):
         return StmtUnrecognized(content=line, line_number=line_number)
@@ -158,7 +160,7 @@ def parse_columns_for_standard_form(line, event_field_length, source_field_lengt
 		    line_number=line_number)
 
 
-def parse_trailer_statement(line, line_number):
+def _parse_trailer_statement(line, line_number):
     trimmed = line[3:].strip()
     return StmtTrailer(trimmed, line_number=line_number)
 

pycmx/util.py

@@ -4,7 +4,22 @@
 # Utility functions
 
 def collimate(a_string, column_widths): 
-    'Splits a string into substrings that are column_widths length.'
+    """
+    Split a list-type thing, like a string, into slices that are column_widths 
+    length.
+    
+    >>> collimate("a b1 c2345",[2,3,3,2])
+    ['a ','b1 ','c23','45']
+
+    Args:
+        a_string: The string to split. This parameter can actually be anything
+            sliceable.
+        column_widths: A list of integers, each one is the length of a column. 
+
+    Returns:
+        A list of slices. The len() of the returned list will *always* equal 
+        len(:column_widths:). 
+    """
     
     if len(column_widths) == 0:
         return []
@@ -14,51 +29,3 @@ def collimate(a_string, column_widths):
     rest = a_string[width:]
     return [element] + collimate(rest, column_widths[1:]) 
 
-
-class NamedTupleParser:
-    """
-    Accepts a list of namedtuple and the client can step through the list with
-    parser operations such as `accept()` and `expect()`
-    """
-    
-    def __init__(self, tuple_list):
-        self.tokens = tuple_list
-        self.current_token = None
-    
-    def peek(self):
-        """
-        Returns the token to come after the `current_token` without
-        popping the current token.
-        """
-        return self.tokens[0]
-    
-    def at_end(self):
-        "`True` if the `current_token` is the last one."
-        return len(self.tokens) == 0
-    
-    def next_token(self):
-        "Sets `current_token` to the next token popped from the list"
-        self.current_token = self.peek()
-        self.tokens = self.tokens[1:]        
-    
-    def accept(self, type_name):
-        """
-        If the next token.__name__ is `type_name`, returns true and advances 
-        to the next token with `next_token()`.
-        """
-        if self.at_end(): 
-            return False
-        elif (type(self.peek()).__name__ == type_name ):
-            self.next_token()
-            return True
-        else:
-            return False
-    
-    def expect(self, type_name):
-        """
-        If the next token.__name__ is `type_name`, the parser is advanced. 
-        If it is not, an assertion failure occurs.
-        """
-        assert( self.accept(type_name) )
-
-

setup.py

@@ -4,7 +4,7 @@ with open("README.md", "r") as fh:
     long_description = fh.read()
 
 setup(name='pycmx',
-      version='0.6',
+      version='0.7',
       author='Jamie Hardt',
       author_email='jamiehardt@me.com',
       description='CMX 3600 Edit Decision List Parser',

tests/test_parse.py

@@ -4,70 +4,98 @@ import pycmx
 
 class TestParse(TestCase):
 
-    def test_edls(self):
+    files = ["INS4_R1_010417.edl" ,
-        files = ["INS4_R1_010417.edl" ,
+        "INS4_R1_DX_092117.edl",
-                  "STP R1 v082517.edl",
+          "STP R1 v082517.edl",
-                  "ToD_R4_LOCK3.1_030618_Video.edl",
+          "ToD_R4_LOCK3.1_030618_Video.edl",
-                  "TEST.edl"
+          "TEST.edl",
-                ]
+          "test_edl_cdl.edl",
-        
+          "INS4_R1_DX_092117.edl"
-        counts = [ 287, 250 , 376, 120  ]
+        ]
 
+    def test_event_counts(self):
+
+        counts = [ 287, 466, 250 , 376, 120 , 3 , 466 ]
+
+        for fn, count in zip(type(self).files, counts):
+            with open(f"tests/edls/{fn}" ,'r') as f:
+                edl = pycmx.parse_cmx3600(f)
+                actual = len( list( edl.events ))
+                self.assertTrue( actual == count , f"expected {count} in file {fn} but found {actual}")
+
+    def test_list_sanity(self):
+        for fn in type(self).files:
+            with open(f"tests/edls/{fn}" ,'r') as f:
+                edl = pycmx.parse_cmx3600(f)
+                self.assertTrue( type(edl.title) is str )
+    
+
+    def test_event_sanity(self):
+        for fn in type(self).files:
+            with open(f"tests/edls/{fn}" ,'r') as f:
+                edl = pycmx.parse_cmx3600(f)
+                for index, event in enumerate(edl.events):
+                    self.assertTrue( len(event.edits) > 0 )
+                    self.assertTrue( event.number == index + 1 )
 
-        for fn, count in zip(files, counts):
-            edl = pycmx.parse_cmx3600(f"tests/edls/{fn}" )
-            actual = len(list(edl.events ))
-            self.assertTrue( actual == count , f"expected {count} in file {fn} but found {actual}")
 
 
     def test_events(self):
-        edl = pycmx.parse_cmx3600("tests/edls/TEST.edl")
+        with open("tests/edls/TEST.edl",'r') as f:
-        events = list( edl.events  )
+            edl = pycmx.parse_cmx3600(f)
-    
+            events = list( edl.events  )
-        self.assertEqual( int(events[0].number) , 1)
+        
-        self.assertEqual( events[0].edits[0].source , "OY_HEAD_")
+            self.assertEqual( events[0].number , 1)
-        self.assertEqual( events[0].edits[0].clip_name , "HEAD LEADER MONO")
+            self.assertEqual( events[0].edits[0].source , "OY_HEAD_")
-        self.assertEqual( events[0].edits[0].source_file , "OY_HEAD_LEADER.MOV")
+            self.assertEqual( events[0].edits[0].clip_name , "HEAD LEADER MONO")
-        self.assertEqual( events[0].edits[0].source_in , "00:00:00:00")
+            self.assertEqual( events[0].edits[0].source_file , "OY_HEAD_LEADER.MOV")
-        self.assertEqual( events[0].edits[0].source_out , "00:00:00:00")
+            self.assertEqual( events[0].edits[0].source_in , "00:00:00:00")
-        self.assertEqual( events[0].edits[0].record_in , "01:00:00:00")
+            self.assertEqual( events[0].edits[0].source_out , "00:00:00:00")
-        self.assertEqual( events[0].edits[0].record_out , "01:00:08:00")
+            self.assertEqual( events[0].edits[0].record_in , "01:00:00:00")
-        self.assertTrue( events[0].edits[0].transition.kind == pycmx.Transition.Cut)
+            self.assertEqual( events[0].edits[0].record_out , "01:00:08:00")
+            self.assertTrue( events[0].edits[0].transition.kind == pycmx.Transition.Cut)
 
-    def test_channel_mop(self):
+    def test_channel_map(self):
-        edl = pycmx.parse_cmx3600("tests/edls/TEST.edl")
+        with open("tests/edls/TEST.edl",'r') as f:
-        events = list( edl.events  )
+            edl = pycmx.parse_cmx3600(f)
-        self.assertFalse( events[0].edits[0].channels.video)
+            events = list( edl.events  )
-        self.assertFalse( events[0].edits[0].channels.a1)
+            self.assertFalse( events[0].edits[0].channels.video)
-        self.assertTrue( events[0].edits[0].channels.a2)
+            self.assertFalse( events[0].edits[0].channels.a1)
-        self.assertTrue( events[2].edits[0].channels.get_audio_channel(7) )
+            self.assertTrue( events[0].edits[0].channels.a2)
+            self.assertTrue( events[2].edits[0].channels.get_audio_channel(7) )
 
 
     def test_multi_edit_events(self):
-        edl = pycmx.parse_cmx3600("tests/edls/TEST.edl")
+        with open("tests/edls/TEST.edl",'r') as f:
-        events = list( edl.events  )
+            edl = pycmx.parse_cmx3600(f)
-
+            events = list( edl.events )
-
+
-        self.assertEqual( int(events[42].number) , 43)
+            self.assertEqual( events[42].number , 43)
-        self.assertEqual( len(events[42].edits), 2)
+            self.assertEqual( len(events[42].edits), 2)
-
+
-        
+            self.assertEqual( events[42].edits[0].source , "TC_R1_V1")
-        self.assertEqual( events[42].edits[0].source , "TC_R1_V1")
+            self.assertEqual( events[42].edits[0].clip_name , "TC R1 V1.2 TEMP1 FX ST.WAV")
-        self.assertEqual( events[42].edits[0].clip_name , "TC R1 V1.2 TEMP1 FX ST.WAV")
+            self.assertEqual( events[42].edits[0].source_in , "00:00:00:00")
-        self.assertEqual( events[42].edits[0].source_in , "00:00:00:00")
+            self.assertEqual( events[42].edits[0].source_out , "00:00:00:00")
-        self.assertEqual( events[42].edits[0].source_out , "00:00:00:00")
+            self.assertEqual( events[42].edits[0].record_in , "01:08:56:09")
-        self.assertEqual( events[42].edits[0].record_in , "01:08:56:09")
+            self.assertEqual( events[42].edits[0].record_out , "01:08:56:09")
-        self.assertEqual( events[42].edits[0].record_out , "01:08:56:09")
+            self.assertTrue( events[42].edits[0].transition.kind == pycmx.Transition.Cut)
-        self.assertTrue( events[42].edits[0].transition.kind == pycmx.Transition.Cut)
+
-
+            self.assertEqual( events[42].edits[1].source , "TC_R1_V6")
-
+            self.assertEqual( events[42].edits[1].clip_name , "TC R1 V6 TEMP2 ST FX.WAV")
-        self.assertEqual( events[42].edits[1].source , "TC_R1_V6")
+            self.assertEqual( events[42].edits[1].source_in , "00:00:00:00")
-        self.assertEqual( events[42].edits[1].clip_name , "TC R1 V6 TEMP2 ST FX.WAV")
+            self.assertEqual( events[42].edits[1].source_out , "00:00:00:00")
-        self.assertEqual( events[42].edits[1].source_in , "00:00:00:00")
+            self.assertEqual( events[42].edits[1].record_in , "01:08:56:09")
-        self.assertEqual( events[42].edits[1].source_out , "00:00:00:00")
+            self.assertEqual( events[42].edits[1].record_out , "01:08:56:11")
-        self.assertEqual( events[42].edits[1].record_in , "01:08:56:09")
+            self.assertTrue( events[42].edits[1].transition.kind == pycmx.Transition.Dissolve)
-        self.assertEqual( events[42].edits[1].record_out , "01:08:56:11")
+
-        self.assertTrue( events[42].edits[1].transition.kind == pycmx.Transition.Dissolve)
+    def test_line_numbers(self):
-
+        with open("tests/edls/ToD_R4_LOCK3.1_030618_Video.edl") as f:
+            edl = pycmx.parse_cmx3600(f)
+
+            events = list( edl.events )
+            self.assertEqual( events[0].edits[0].line_number, 2)
+            self.assertEqual( events[14].edits[0].line_number, 45)
+            self.assertEqual( events[180].edits[0].line_number, 544)