ARRI Raw Dailies Workflow

This article first appeared in Definition Magazine, UK, in 2011.


ARRI Alexa

What were your thoughts when you first discovered that you were going to be taking an ArriRAW film through dailies?

Sixteen19 was involved with some of the very first ArriRaw jobs anywhere. We shared the workflow execution with another company on Extremely Loud, Incredibly Close in New York in 2010, and then set up Gambit soon after in London.

The Codex recorder had just been certified by Arri to be used as a recording device for ArriRaw, which was a significant development because Arri themselves hadn’t made a recorder for this format- they were relying on a secondary market of companies to build accessories for the camera.  There are a few factors that go into investigating any new format,

For feature film work the first thing you need to find out is what the format is doing in it’s colour space and debayering. Basically, how the raw data will be processed in order to look like film.

In simple terms the colour space is represented by a ‘colour curve’, which basically describes how the image colour  tonality is represented within a digital image. Sony has S-Log, Panavision has PanaLog, and Arri has ArriRaw- each of these are manufacturer-specific ways of encoding and representing a digital image. These in turn almost always reference Cineon encoding, which was the original file type that Kodak developed to represent a film frame as a film scan.

The debayering really just means how software further down the workflow will interpret the raw digital image data into a coherent picture image.

Once you’ve worked out how these two elements work together to produce an image out of the camera system, then you are ready to try and use these things to give the DP the look he is after on the dailies, either through a DIT colour grading on set, or by having a dailies colourist go a more traditional route and do this work after shoot is finished.

It’s best not to lose sight of what the dailies pipeline is for- it’s not just about creating pretty pictures, although that is the most immediately obvious role that dailies have. Doing the daily rushes on any film is as much about organising the material that’s coming from set, so that it can be coherently used by an editor and post production team for sometimes four times as long a time period as the shoot. So the hidden side of any new camera is working out as well just how you can get as much information from the process as possible. This is called the metadata, the information about the data you hold.

How much involvement did you have with establishing the on-set workflow? How important is collaboration between DIT and DoP on the one hand, and equipment on the other?

In my role as Workflow Supervisor, I usually kick around the workflow between the DP, editor and producer/ studio for a few weeks in Pre Production. The DIT usually comes into that conversation as well, which is a factor. There are also tests that need to be run first of all from the set to dailies/ editorial, then right the way through to VFX plate delivery and the digital intermediate finishing for the feature. It’s a mix of technical knowledge, equipment product knowledge and politics. The diplomacy and politics can measured exactly between the requirements of the principal creatives and the shortfalls of any budget.

The collaboration starts with how the DP wants to work with a DIT, which can range from a simple monitoring setup that shows people on set the camera images, and can grow into a whole grading setup ‘in the tent’, with full colour management and image control.

This collaboration also extends to the editorial crew, in working out precisely how they want their files collected and organised. The quality control (QC) pass to check each file is intact, sound synching to picture, and laying everything down into the correct bins and structure are all negotiated with the editor and crew.

And finally the archive is made, which is arguably the most important process of all, because this is where the real asset is collected- it’s stored on tape called an LTO5 tape, and is akin to the exposed negative of a film shoot- treat with extreme care.

The job of Workflow Supervisor is to make sure that the transition between each step is managed and controlled to ensure that every frame gets to editorial and onto an archive tape.

How did you ensure monitor calibration throughout the chain?

You hire a colour scientist to generate the required Colour Management Look Up Tables (LUTs) and calibrate monitors to the required specification. This used to be real voodoo, but now comes down to standards that can be pulled from software. The Arri website even has quite a good LUT generator that can make you LUTs for a few different colour management systems.

It’s probably just as important to ensure that all of the key monitors around the set and dailies environment all match in terms of calibration. Everyone looking at a particular monitor that’s out of alignment with everything else can cause quite a few problems down the chain, not the least of which is the confidence in the pipeline itself. Confidence is sometimes the hardest thing to get back on-set, and so again this work should be done during the camera testing period.

This is one of the most important things to stress. The advent of digital cameras trashed the notion that there was any ‘one way’ to execute post production on a feature film. It has actually done the opposite- given filmmakers an unrivalled set of options for post, to improve the production values of the film, so it can compete with films that have double or triple the budget. The camera tests before shoot are no longer just camera tests- they should be extended into the teams that will execute dailies, vfx, and the finishing of the film. One doesn’t have to lock down every detail of every shot before the first day of shoot, but the whole process should be described, agreed on by heads of department and tested before the first day of shoot.

The material came in to you via Datapack – a simple process?

Fairly straightforward. It’s basically just like copying any other data- it just takes time. After a few tests, you start to get a feel for the time factor- 2x runtime, 3x runtime, and this should be communicated to all so people have an idea of the timing.

The more important process is cycling the mags back to set. Because a film’s budget can only afford an allocation of a set amount of mags, and the camera department shouldn’t have to stop filming just because a mag isn’t available. It’s not just a matter of copying and sending back, however, as the digital assets from that datapack should only be cleared when the archive has safely written out to tape- only then is the asset ‘secure’ and can be

How did the dailies workflow operate?

The private joke about workflow diagrams is that the actual work isn’t in the bubbles, but in the lines.

2013 ARRI RAW Workflow document.

Any special considerations that had to be taken into account due to using the ArriRAW format?

They are not special considerations, but people need to be aware of the file size- ArriRaw is a large, high bandwidth, high latitude file, and so takes considerably more time in data movement and  processing Pro Res or smaller compressed files. Also that there are subtle differences between the video line out of the camera that you use for monitoring and the actual file that the camera records. During your testing phases with the DP and crew, you should be able to describe this and factor it into your working methods.

And finally, say I’m coming to you with my latest movie and I’m planning on shooting it in the format, what are your three top bits of advice?

Give plenty of time for testing during your camera tests – don’t leave it until first day of shoot.

Consult not only with the DP, but also with the editor and post supervisor down the chain- insist on a workflow test from the camera tests right through to a conform of the data at your DI house, that takes into account all the stages and people that the data has to pass through.

Don’t have one person do everything – this may be the cheapest way out, but once shooting ramps up, you’ll regret your decision when that guy is hospitalised due to working for days straight. It’s not pretty…

Rise Of The Workflow Producer

This article was first published in Inside Film Magazine, Australia- (http://if.com.au/), 2011

 

We landed through Heathrow Thursday, and spent the Royal Wedding Weekend in an Ealing Studios rehearsal room, booting up gear, pulling cables, software updating, and going over the last minute things we needed to buy the next day to set up. The keys to the cutting room came late, and we had a full day’s and night installation to get done to make it to pre-shoot day. Besides, we’d promised the editor it’d be up and running Monday. Our trucks arrived with two burly South African removalists, and two on-loan runners from Technicolor turned up with a trolley. Soho is no joke with the skinny stairwell, and we had to disassemble our server half way up just to get it up and around a corner. We’d moved in on that first day with two mac towers, two avids, a server, and we had a 50 day shoot starting the next day. This is how films are now made.

Earlier this year at the Hollywood Post Alliance Tech Retreat in Palm Springs, a getaway conference for film technologists and the post community in LA, the shift was palpable. In the wide selection of speakers and panelists, everyone was talking about their own, special, bespoke, unique, ‘snowflake’ workflows and practices for digital shooting- no two workflows are the same. The proliferation of digital formats, of commodity computers running mass-market software and yet another generation of digital cameras coming through the ranks have split at the seams the 100 year traditions of filmmaking, and people are finding all sorts of innovative ways of working that embrace both ‘film discipline’ and the responsiveness, flexibility and innovation of digital capture.

Locally, we hired a colour scientist who was able to handle all of the colour management for the Alexa camera through a £600 plugin for shake on his Mac tower. He had worked at a larger VFX house for a few years, but prefers freelancing now and is in demand by smaller workflow companies such as ours, who hire him by the week. We worked with 4KLondon, the Digital Image Technician agency, who were able to match us up with an excellent freelance guy who was our eyes and ears on set, who chased the director of photography around with his cart, showing him his beautifully shot images in realtime on a calibrated monitor, and determined the colour balance that the DP wanted. And we set up ourselves, to pull this all together, to create the dailies and most importantly, the data archive of the digital negative, the primary asset of the production.

The standardisation of 35mm and 16mm for shooting and printing formats in the early 20th Century was a boon to the business of distributing films. It meant leaving behind many formats (9.5mm! 26mm Freise- Green! Dufay Colour!), but gave an economy of scale for the business. Digital shooting has been a wild west for a little while now, with every camera manufacturer working in a slightly different colour space, resolution and proprietary format. This year at HPA the Academy Of Motion Picture Arts and Sciences introduced to the world their IIF- ACES colour workflow for digital cameras, which allows manufacturers to still maximise the depth and precision of their images and sensors, but provides clear pathways for the careful handling of those images in post production. IIF-ACES is an excellent attempt to at once institute a standard of colour management and reproduction for digital cameras, and at the same time still allow individual camera companies to push the boundaries of what their camera can do.

The customisation of the camera and it’s images within the boundaries set out by the standard still gives an enormous range of creativity and technical calibration for the technician and artist. And as the talent that used to be locked up in VFX and Post houses becomes freelance, smaller, more lightweight ‘insurgent’ teams area assembling all over the place to get films made. If I worked in a lab, I would say I was worried.

After 10 weeks, we’ll move to New Mexico. Well, I say move: The dailies team will stay here on the 3rd floor in Berwick Street, a block up from the markets. An operator that we know in LA will go across to New Mexico, where we’ve booked a high speed uplink, and will upload the ARRI RAW files directly to the dailies & cutting room in London after shoot every day.This bandwidth is getting cheaper all the time, and satellite can be booked like you’d order a book from Amazon, and is certainly much cheaper than sending the data across state lines to a lab in California, or flying the dailies team into New Mexico. After they’ve shot for a week, action will resume in London. We’ll be all done and gone in 11 weeks, and moved onto the next job, in another location, with another format, another workflow and different problems to solve.

With so many different formats and options, how does one production actually ever get started? There’s no one industry standard workflow anymore. HPA was abuzz with talk of how this works, and a new position that is gaining precedence: the Workflow Supervisor. This is the person who would be able to deal with all of the technical and creative issues when deciding to shoot digitally. They would liaise between the DP, the editor, producer and director, and would have to support each position with specific knowledge, and a very strong communicated idea of the way everything fits together. It’s a strange mix of networking IT, Avid support, cinematography, VFX and production skills embodied in one person, and these people do exist. They would not only take a film right through from the camera tests to the finished Digital Cinema masters technically, but also be able to assemble small and medium sized teams where ever they are, to get the job done. Some of these people are still directing a facility to the proper workflow, but more and more the workflow supervisor is responsible for setting up and managing the workflow from conception to build to execution.

Naturally there are only a few of these people at the moment, who can just pop up a shop, work together for a short amount of time, and then disband and go do something else, but it’s the same as people who crew on set in shooting production have been doing most of their careers, working in groups that assemble just for the film at hand. Now that digital pipelines have freed the brave to venture out from beyond a facility’s walls, they can see that their skills and experience are very valuable and can slot in to a production.

And leading this whole team, putting it together, planning the pipeline, executing and getting the film started and then maintained, is the Workflow Supervisor. If you’re planning a digital shoot, and you haven’t talked to anyone yet who can perform this role for you, you’re nowhere near ready to dive into the shoot. They will not only save you money and time, but they’ll get your film made while they are doing it.

Next month it’s Pittsburg, and then a film in Vancouver. The summer may be a surfing film in Northern California, but by November it’s definitely Alaska for the teen comedy. It’s a way of working that has been known in the film industry, but not by the technology people on the back end. The mobile sector is a small but rapidly growing sector of the film industry with a few companies jockeying for position and market share. With contraction evident everywhere else in the big-ticket end of film technology, a light-weight, scalable, movable and robust service that will be whereever the film is, that works directly with the filmmakers, without expensive premises or million dollar kit, has got to be a way forward.

– See more at: http://www.sixteen19.com/blog-entry/rise-workflow-producer#sthash.gexmjDvt.dpuf

Noone’s Going To Bring You A Hard Disc

Extending on from the last post, the Old Post businesses actually can see the threat that’s posed from the New Way of working, but wonder how they can get involved to protect their market share. Generally, the approaches are the same, but prioritise the Old Post systems and ways of doing things to the detriment of the New Way.

An example: Many managers, in thinking about how to deal with digital cinematography, have looked at the way their existing business is structured around film dailies, and in their mind have only taken the leap as far as exchanging a film can with a hard drive, but everything else staying the same. The trouble is, once you swap film formats for digital acquisition, Everything Changes.

How many times have you seen the following press release in the last few years:

Facility X is adapting it’s processes to the new digital shooting technology, and now would like to introduce their new Data Lab/ Digital Dailies/ Data Processing/ Digital Data Lab Dailies Processing service to the local market. It works exactly the same way as a film lab works- all producers need to do is drop off a hard disk of dailies before11pm every night, and send a runner around the next morning to pick up another hard disk of editorial files and your DVDs. It’s That Easy.

Hey Good Lookin'..
 Fundamentally, this is the Old Post model tarted up a bit to feel sexy. It doesn’t realize yet that:

  • The images are no longer intrinsically tied to the format. This is the most important thing to understand, and that few traditional places get: Unlike 35 or 16mm film, the hard disk doesn’t actually determine characteristics of the digital file- it’s just the transport mechanism. A transport mechanism that can be interchanged without loss of quality of the images themselves. Begs the question: nowadays, with internet speeds what they are (and even once-prohibitive satellite now being relatively cheap) why do you physically have to take anything anywhere? And do your bond insurers really want a hard disk getting around town with your images on it? Really?
  • As above, why is your material being delivered back on a physical format? Why on earth are you still making crappy DVDs with burnins to evaluate your images during the shoot? If you needed a physical format at all, why wouldn’t you be able to download a disk image that’s relevant, and burn it on your own laptop?
  • With the cost of the equipment dropping to near commodity prices every day, what’s stopping traditional post operators getting out onto set and working directly with production. This person can be processing the images and handing to your editor straight away after being acquired onset. With a good pipeline designed beforehand an editor can be cutting a few hours behind shoot, which will lower reshoots and continuity problems, and increase shooting speeds. I’ll cover some of the great hardware that’s coming out now for this purpose in future posts.
  • Why is it an overnight service? This was typically a chemical-lab imposed deadline, because you needed to chemically process everything in a linear fashion all at the same time, and you needed to impose a cut-off so that everything could be delivered and then be set off for the evening. With digital processing now able to be done at least at runtime, why do you need 7 hours to ‘process’ two hours of dailies?
  • Asset management and online services are usually an outside service that is an optional extra, and not at the very core of the system. It’s not about the processing anymore, if it ever was, because everyone with Final Cut can process. It’s all about accessibility now, and online should be at the centre of the service, not at the periphery.

 Of course,  RED users have been doing this for years with the DIT position on set, but this near-set or cloud workflow is now breaking through to become available for higher-end cameras as well as affordable for indie productions.

Not everyone wants to own gear at the end of their show, but still want to work in the new way and there are a variety of businesses now emerging to compete with the existing players on their own terms. As just one example:

I’ve had a few chats with Michela Ledwidge from Rack & Pin (@rackandpin), and think that these guys show that they understand the new environment for post, which isn’t about physical facilities and gear, but networked cloud computing. At their heart, they refer to themselves as a “Cloud service for media productions”, and focus on the client’s interaction with their own material, not the processing. Sure, they have a physical address that you can still drop your hard drive into if you want, but it’s not the office- they find partners in existing storefronts that upload onto R&P’s cloud based server system. And their business isn’t just the walled garden of dailies- as the website says, it’s ‘Useful Screens To Manage Your Data’. Within the one system, they can service dailies, vfx approval, production management etc, and even offer to create bespoke applications and workflows for their clients, so that as new business models emerge from an increasingly digital approach, they can adapt and scale accordingly.

They don’t have one hard product to sell you, developed around existing film infrastructure they are still madly trying to keep relevant. They are mobile, adaptable, in the cloud, and are focused more and more on people managing media directly as a subscription service where ever the clients happen to be. They and businesses like them are going to have an increasing impact, especially on the Old Way businesses waiting for your hard drive, and on an emerging filmmaking community that is looking for web-based secure services and has grown up with online services.