A technologically inclined Star Trek fan known as “Captain Robau” was experimenting with using machine learning algorithms (which is commonly called “AI” in marketing buzzwords) to upscale art assets on the video game Final Fantasy VII. He had such good results with it, that on a whim he decided to do some tests on his favorite Trek show, Deep Space Nine. There are full details on his blog, he also plans to do a follow up with even more technical details in the coming days. The results show promise, and we’ve embedded a few examples below.
As many fans know, there are two Star Trek shows stuck in standard definition, Deep Space Nine and Voyager, and there’s no indication that CBS will do what they did with The Next Generation, a total remaster from ground up, due to high cost and changing markets.
Like many fans, Captain Robau has lamented the fact that DS9 is stuck in the 1990s, but then he thought of using AI upscaling based on his experiences with Final Fantasy:
Just like Final Fantasy 7, of which I am upscaling the backgrounds, textures, and videos in Remako mod, DS9 was also relegated to a non-HD future. While the popular Original Series and The Next Generation were mostly shot on film, the mid 90s DS9 had its visual effects shots (space battles and such) shot on video.
While you can rescan analog film at a higher resolution, video is digital and can’t be rescanned. This makes it much costlier to remaster this TV show, which is one of the reasons why it hasn’t happened.
This is where neural networks could come in, I thought. With tools like AI Gigapixel, I knew it might be possible the low definition frames of DS9 can be scaled up to a higher definition such as 1080p or 4K. It would never be the same as proper remastering, but it would a step in the good direction.
For a first test he tried out still frames from the the beloved episode “The Visitor.” Click to open each image in a new tab, so you can see the differences in details for yourself.
Original image from “The Visitor”
Upscaled image from “The Visitor”
Testing the process on actual video
Captain Robau then decided to use the opening five minutes of the fan favorite season six episode “Sacrifice of Angels” to test the machine learning’s algorithms on moving pictures, not just still frames. You can see the machine learning upscaling at work below in a side-by-side comparison video. Be sure to watch it full screen and at 1080p.
DS9 opening credits in UltraHD
And to get really ambitious he tried out upsampling the opening credits to 4k. As the author notes on his blog:
This nearly melted my computer, as it is a lot more intense to upscale than 1080p so I’ll stick to this single video for 4K examples of DS9 Enhanced.
Not a substitute for a proper remaster
As Captain Robau notes, this is not really going to achieve the same results that a proper remastering would. Upscaling is making educated guesses at deriving higher quality image data from a lower quality source. The example videos show just how variable the end results can be, in some scenes there’s a noticeable uptick in detail, and others it’s nearly as soft. That’s primarily due to the fact the source is simply lacking information to do much with. The source is still limited to the poor color (chroma) and brightness (luminance) information inherent in recording things to NTSC video tape. There are also artifacts from the early CGI that can be seen, since they exist in the source, they’re passed on to the upsampled copy too.
While the results can be quite impressive, even with the best algorithms at work, the old saying “garbage in, garbage out” still holds true. While not a perfect apples-to-apples comparison, check out the AI upscaled image of DS9 and compare it to when the space station was shown on TNG in season 6 as part of “Birthright,” Next Gen of course was given a full remaster that started in 2012.
The Deep Space Nine model has a ton of detail, much of it was not clearly seen before Next Gen was remastered. One of the most obvious reveals from the remaster is that there’s periodic yellow detailing on the station, which was previously lost — turned into a muddy shade of grey in standard definition. Because the AI upscale only has the old standard definition version to use as a reference, the yellow detail is not present.
AI upscaled footage of the space station
CBS remastered footage of the space station from TNG “Birthright”
What’s next?
Captain Robau has made it clear this was just a test and he has no intention of trying to apply this type of process to full episodes. And for the reasons outlined above, it is clear that this process would not – and could never – achieve the kind of results seen with the Star Trek: The Next Generation remastering, which was a meticulously labor-intensive project. However, the tests show that marginal improvements could be made to the standard definition version of the show. Feeding the learning algorithm more examples and, ideally, higher quality images too, could in theory help narrow the gap a bit as well. Perhaps in the coming years, it would be possible for CBS to use a process like this for both Deep Space Nine and Voyager, which would be significantly less expensive than the process used for TNG. This enhanced product would not really be a true HD release but could add value for syndication and streaming sales, especially in the years to come.
I would truly be more interested in a DS9 restoration on CBS All Access than another new Star Trek series.
Love your screen name….
Yup, I’d be totally down with that. (and i’d buy it too). Also, totally love the screen name.
Not perfect, but not bad. The videos looks solid enough on my phone, at least
The visual effects look pretty good in HD. It needs a bit more work.
The opening credits sequence looks beautiful, but then again, that Seasons 4-7 redo of the opening always looked nice. It’s Seasons 1-3 that looked rather rough in my latest re-watch of the series.
At least someone’s trying, though!
I guess I wasn’t paying attention all this time. I thought they were going to remaster all the footage they were going to use in this documentary. I feel so let down right now.
This article is not about the documentary. This is about a fan playing around with some cool new image processing tools.
As we’ve reported previously, the DS9 documentary is indeed remastering all the footage they’re using.
It’s a total restore, remastering is something different.Remastering is different software tools used, like clean up , sharping, and DNR. A total restore is going back to the original elements redoing the FX like they did with TOS and Next gen. The cost is thru the roof and why were getting clips restored with the documentary. So again because of cost we will never see DS9 in true HD or any other series after.
CBS and Paramount sailed that ship along time ago and TOS and Next gen didn’t sell that well on blu-ray. In fact you can get them dirt cheap right now. I paid for the first season of TNG on blu-ray for 14.00. and most other seasons are going for 29.00 bucks at the most.The only way that would happen is if outside sources giving money could make that happen, but CBS and Paramount would ask ok , but what’s in it for me.Another words forget it.Enjoy your DVDS
Not sure why you felt the need to comment repeatedly on something that is a matter of semantics.
Restoration, and Remastering are two sides of the same coin. They are used synonymously (if perhaps technically incorrectly, but that is debatable), even the industry itself uses them them interchangeably. Regardless, in the case of TNG-R (and any potential DS9/VOY versions), remastering is the correct term.
The term Restoration has its roots in the analog domain, when film restoration was done without computers, photochemically making new versions. Today it generally refers to when a film is scanned to digital for preservation and a modest amount of basic clean up is done, typically dirt and scratches are cleaned up. Also typically, but not always, this also involves a new color grade from an authoritative source (if they are is still alive they use the cinematographer or secondarily the director, or if no one is available they use production notes and other sources to establish the grade). In fact to some purists, doing anything but digitally scanning a film (not even doing touch up work) means you’re now “remastering” it.
Remastering is a term that gained prominence when digital tools came in to the film restoration process, so it can mean anything from a basic restoration to much more complex changes. It often, but by no means always, refers to when a project goes beyond basic preservation, and can include things like: reassembling missing pieces for a new cut, more intensive clean up of wear and tear in the film stock, etc. In the case of something like TNG-R the term remastering certainly applies, since it was rebuilt from the ground up. It was like doing the post-production on the show all over again. So quite literally there are now new digital masters for TNG.
The remastering term also applies to TOS-R since it was both restored, and then as an option, alternate new VFX were created.
Matt Wright,
I don’t recall where I got this notion, but some source lead me to believe that while the old VFX was rescanned, it wasn’t “restored”, i.e. scratches removed, cleaned, color corrected, etc. I was of the notion the budget for that was eaten by the new VFX?
Please, update me if more was done than I thought I knew.
My understanding is “everything” was cleaned up, but updating the VFX was the primary motivation or justification for the effort.
So if you’re watching with the original film effects, those portions from the original negative have also been cleaned up for HD capture.
What was NOT done was taking apart and recompositing the original effects, as was later necessitated for TNG project. Those raw elements probably wouldn’t exist anyway for TOS. This is why the new dissolves are awkwardly timed, so as to avoid capturing any portion of an original dissolve to a film-composited special effect.
@Sam, Right the original VFX elements were long since trashed. Those kinds of things weren’t considered worth preserving back then.
I don’t recall where I got this notion, but some source lead me to believe that while the old VFX was rescanned, it wasn’t “restored”, i.e. scratches removed, cleaned, color corrected, etc.
They’re a mess. Plain and simple, I don’t think they spent much time on them because of the mandate to make new VFX and the terrible shape they were in.
Those elements are basically beyond fixing, no amount of clean up would really ever make them look good (especially so in the time and budget allotted), which is why they wanted to do brand new total replacements for those space shots.
They reused those same stock shots again and again, and every time they did the wear and tear got worse and worse. Because of course it was done in the old method of splicing in the footage again and again where it was needed. Generational loss with film adds up quickly too.
Matt Wright,
That was my understanding.
Although, in regards to generational loss couldn’t they have just gone back to the footage in the first episode where it was used and digitally copied it forward into the later episodes to minimize that?
For that matter, and we both agree an effort like this was not done for the old VFX footage, they could’ve taken each of the scans from episodes where it was used and compared each frame to its twins to see which frame looked best to reconstruct it into a reel of the sequence made from the best of the frames, but, alas…
sadly, TOS-R was done on a tight deadline and shockingly low budget. Sometimes they were cranking them out only days a head of when they were due to be syndicated that weekend. So their mandate was to focus on the live action and new VFX.
Matt Wright,
Indeed, but, to wax “if only” further about it, on the blu-ray I assume they had to install all that branching to be able to switch between the old and new vfx sequences. It shouldn’t have been that difficult to to reroute which ever episodes shared an old VFX to a better version, if an alternate was there on the disc…
If you have a chance read Marc Cushman’s brilliant these are the voyages there were in fact several special effect houses that worked on the original show and the costs of even a transporter effect ran into thousands of pounds even the in 1960 money equivalent of the special guest stars salary back then hence why often later in the series you see Kirk and co board the transporter and then cut to the planet without the effect. When composing the famous opening SFX intro of the show the companys man that handled the effects suffered a nervous breakdown due to the labour intensive job which it was and when gene Roddenberry saw the completed footage he was horrified as very little was usable which is where clever editing came in to save the day! Obviously if you compare star trek to 2001 a space Odyssey it’s like night and day but Kubrick who my great uncle had the pleasure to work on 2 of his films had a huge budget and spent I believe more than 2 years filming such was Kubrick’s eye to detail with sets etc if there was minor imperfections he had the set scrapped and redone the end results speak for themselves and Kubrick ensured no one else could use the models or sets in any other film by having them destroyed! As to the current state of ds9 picture I tried watching an episode the other day and the picture quality was a bloody mess.As a huge fan of ds9 which I rate the best of all trek series it’s a disgrace the money has not been spent on getting these shows out there in hd regardless if the product recoups it’s costs as they are protecting there property/ commodity for future prosperity! .if star trek was owned by Warner ,Sony or 20th century fox who spend millions on preserving and restoring there films I’m sure we would have seen it done.
Chris,
Except for the Cushman reference, I agree with pretty much everything you said. Unfortunately, that Cushman endorsement is enough to discredit all that follows. The guy has no business whatsoever working as a journalist, or creating this faux ‘trek historian’ label or profiting as he had in very dubious ways with these books, which are rife with errors and deliberate misreprsentations, which manifest further when challenged by actual scholars. I’d direct you to any trekbbs thread on Cushman to see links to the expert refutations by HARVEY and others.
Re: Cushman, kmart beat me to it.
Also, I’m well aware of the time/effort/costs of making TOS VFX. I’m a huge TOS fan, and have been reading about how TOS was made since my first beat up garage sale copy of The Making of Star Trek.
Matt, reading that reminded me that I misread TMOST first time through in 7th grade, and I thought they physically cut a rectangular hole in the film to ‘matte’ in the starfield on the bridge viewscreen. That was about 300 reads back …
I thought they physically cut a rectangular hole in the film to ‘matte’ in the starfield on the bridge viewscreen
lol, I could totally see misunderstanding that. It’s not obvious unless you see an example.
Uh, this has nothing to do with the documentary.
Every clip of DS9 in the doc is being remastered in HD. They can’t likely crowd-fund the entire series (the way the coming movie was). Don’t be too disappointed. It may well be the spark that gets DS9 remastered. You never know. 😊
I’m excited to see it.
Clips are restored not remastered and we will never see the entire series in HD, money talks and profits even more. The TOS and Next gen did poorly in blu-ray sales TOS did better but next gen flopped in sales.I get it from a business standpoint. The cost was thru the roof to do those 2 series. You can’t expect a company to justify that cost when the profits won’t be there. To be fair the cost of one season was high, but the cost doing those series was real costly. Its a business and we as fans would want that, but unless they know the sales will be there at their price point ,forget it.Besides they say physical media is down , I don’t believe that but that’s what they think.
What would have been nice is if they’d gone ahead with the HD-DVD + DVD ‘flippies’ for Seasons 2 and 3 of TOS after Toshiba pulled the plug on HD-DVD. At the least they should’ve made as many HD-DVD sets as they did for Season 1. Everyone who bought a Season 1 set would’ve wanted 2 and 3 to match. But instead, everyone who wanted all three seasons in HD had to wait for the Blu-Ray release then buy Season 1 a second time.
Gregg E.,
Why buy Season 1 again? Was their some sort of Mission Impossible self-destruct of the HD DVD players and discs because my player still works?
Gregg E.,,
“Was their …” should be “Was there …”
You’re in for a rough life if that’s what lets you down…
Control is real.
This is exactly what I was talking about in a previous article, that I got shredded and insulted for. While this isn’t true HD, there may be a day coming when the same kind of tech can achieve better results, for a fraction of the cost of the kind of remastering required today.
What you keep claiming is that there’s an automated way to do a full remastering. This is advanced trickery, you’ll note the errors/poor color fidelity/etc. in the source are of course still reflected in the upsampled version.
If you think I meant fully automated, like with the press of a button, you’re foolishly mistaken.
I don’t know about fully automated, but as there’s one complete episode of DS9 rendered in HD, I would imagine this AI could learn something about rendering the FX models and static sets details from the NTSC masters more accurately from that – not to mention whatever the documentary manages to produce in HD.
Folks, it is still a matter of trying to get a good quality 35mm blowup from a 16mm print — while the original 35mm material is left unexploited. You can project the image through a screen door to cut the grain and use a film with different contrast to give the illusion of a sharper image, but unless you go back to the source material, the results you get are going to be markedly inferior.
Does it look better – most of it, yeah. But it is a ‘I’m squinting at the screen and it looks sharper now’ way, rather than an actual gain. Anytime I have to watch something on VHS (because there aren’t any better versions available, like LOOKING FOR MR. GOODBAR, which was HD on Amazon many years ago but never anyplace else), I have to tweak the settings on the TV like crazy to get the image to a point of being even slightly watchable (and in the case of GOODBAR, that is a movie shot by a genuine master, William Fraker, so the less than ideal image stands out like giant sore thumbs.)
And yeah, I’m pretty sure that in the old discussion about this, that AFTERBURN was expecting that with time, some kind of magical wondrous automated process would likely emerge, because as I recall, that is why I jumped into that thread too. New tech doesn’t automatically equal better tech; the results here, while interesting as an experiment or novelty, don’t seem to point the way to a satisfactory global answer on DS9.
This up-scaling is cool, but the idea of upscaling the episodes this way in any official capacity seems like a half measure and a waste of time. I suppose if it’s the best we’ll ever get, this might be worth pursuing as even with it’s faults, the few shots I’ve seen so far improve some of the “soft” features of the current transfers.
I haven’t seen anyone else touch upon it, but I wouldn’t be surprised that in a few years it will be possible to use “AI”/machine learning to take *all* the original film elements (once they are all manually scanned) and then have the “AI” find the exact scene and take (and all the individually filmed elements) and compile the exact film elements down to the exact millisecond the original episode had.
From what I gathered, the TNG remastering team had to sort through everything manually, finding all the specific takes and edit each of those it all to match the originals, largely by themselves.
I would think that even further advancements in ML could help craft new CGI elements using existing lower-res versions, probably similar to the upscaling featured in this article, but even automating all filmed elements could free up time and resources to dedicate directly to the CGI portion.
Yeah I think some amount of machine learning help with the editing of the raw footage is certainly on the horizon.
When editing an episode, they pulled up the tape dailies and actually overlaid the new scans on it, and used tools that helped them match the shots. That kind thing could be used to lay the groundwork to have machine learning help edit.
https://youtu.be/dPHP5izB8MU?t=480
You’ve hit upon two big time sucks that no amount of AI will help with, because it’s all human labor and judgement calls.
1. “Once they are all manually scanned.”
That is correct, so there is still huge amount of time spent finding all the correct reels and scanning them in.
2. “The TNG remastering team had to sort through everything manually, finding all the specific takes and edit each of those it all to match the originals, largely by themselves.”
That’s true, and I believe that will still be true, more than people might think, even with the AI help for editing. When doing TNG-R, the team found that takes were often in the wrong place. Sometimes this was due to the fact the final take that was used in an episode was a reshoot, so it ended up in the next episode’s reels, or if it was a stock shot — the film was stored with an episode that use it later, and so forth. Often they’d scan in all that they had for an episode and notice a shot was missing. Also there’s the occasional problem of only having an alternate take available, the one used in the final episode is just simply missing. A human will need to make that call.
If AI is progressing the way that they’re claiming (even speaking towards what AI can already do in 2019), it seems like a no brainer for AI do the heavy lifting on a project like this, perhaps even “right now” and not 5-10 years down the road.
I guess it might come down to how many of the reels had to be scanned the way they did with TNG-R. If they had to scan almost all the reels anyway to sort through the takes, I would think scanning it all and letting the AI sort it all out would save a significant amount of time. I suppose there could also be a middle ground where humans make the determination of what reels were likely used and then see what AI can do with it and then do more work later on.
Presumably, the AI would be able to sort through everything on all the reels, including any reshoots or pickups in later episodes since it should be able to match up shots. If there was any kind of question on what take should be used, a human could intervene in the process.
One of the biggest upsides I see here is that the AI could essentially run 24-7, so long as there is material to sift through and edit, it can be “working.”
I could see this helping reconstruct extended versions of episodes that might only have a copy that exists in low-res condition, a la “Measure of a Man.” As long as there’s a source to base the edit off of, the AI should be able to do most of the work. If any production notes involving cut scenes exist, I’d be interested in seeing if the AI could construct those episodes before they were cut for time (obviously as alternate versions of the episodes, and only if the director/producers approved of these alternate/extended versions actually seeing the light of day). Obviously this last part is just an interesting thought, and certainly only something I’d endorse after any actual remastering/editing was done on the broadcast versions.
There’s a bit of extra “quality” to be had in the original SD video, but not much. Noise can be reduced, a finer layer of grain can be added, edges can be sharpened a bit, fancy neural net upscaling can be applied, but the returns diminish very quickly. You just can’t create more actual detail without rescanning the film.
With current technology, no.
20 years from now. Maybe.
“You just can’t create more actual detail without rescanning the film.”
Yes you can! Check out some of the latest ML image results and you will be very astonished!
I look forward to time when AI Will be ale to transform original series or even the animated series into Discovery style of look :D
Jonathan,
Just turn down the brightness, add visual distortion by watching while holding a cigarette lighter and a piece of blue cellophane in front of your eyes, you’ll get your DISCOVERY look to ruin the earlier series just fine.
What are you talking about?
kmart is making a joke about the DSC style. DSC is quite blue in its color grading and likes lens flares.
VS,
ML is what exactly? Is it some machine-vision style interpolation? If so, I don’t think the results would be consistently satisfactory at all.
I mean machine learning, and GANs specifically. Granted, the technology is not there for *universal* application yet, but the basic idea is to feed tons of data (DS9 high res pictures, in this case) and then the algorithms will insert the necessary details based upon these inputs. They would add the yellow stripes to the station, for example, instead of doing a simple edge enhancement. These algorithms have improved so tremendously just in the past five years, that I don’t see why in another five years AI-assisted up-scaling of TV series won’t be feasible both financially and quality-wise. Check out letsenhance dot io for the beginnings!
Can’t edit anymore – the link is letsenhance dot io
No you can’t .When blu-ray first came out they tried rescaning a film or use an old DVD master and the results were horrible. As they say, you can’t polish a turd. You have to go back the original 35 mm film and in the case of Next gen they had to redo the FX because they were done on video tape.Companies don’t think ahead. Christ we have 4K tvs and there pushing 8K already. Some TV networks at least locally are broadcasting in 720p!!CNN 720p where NBC 1080p. No 4K except netflix and amazon prime so why should movie studios think any different. They could care less.
“When blu-ray first came out they tried rescaning a film or use an old DVD master and the results were horrible”.
We are obviously talking about the near-future and not the past… When Bluray came out, that’s like ancient history for technology!
Apparently there’s redundant copies of frames which could be used to reduce errors:
https://www.newscientist.com/article/mg12717313.900-technology-dallas-becomes-quality-viewing-/
True ,but again cost ,sales and profits are everything with these companies. As much as we love Trek and these series I don’t believe the market is there. The best thing to do is to let CBS and Paramount know we want this series in HD. If enough people do that maybe , just maybe they might do that. But as far as we know just to get these clips in HD they had to get a go fund me thing going, what’s that tell you ? As much as CBS and Paramount say Star Trek is there cash cow it’s BS. If it was this series would have been in HD along time ago.
Thomas P Vinelli,
Re: what’s that tell you ?
That if lower-tiered Hollywood studios, like Paramount and CBS don’t get creative in their production financing and take advantage of such new financing possibilities such as Go fund Me, they ain’t going to be here.
Deep space nine is fine the way it is. I would rather see Voyager get redefined imageries
Get in line. :-P
Me too! Would love to see Voyager get the treatment
No what really needs an overhaul is TAS !
Albatrosity, I so agree with that. Filmation did the best they could, but lack of facial expression, and stiffness in general really takes it’s toll. TAS could potentially be transformed into something that would look MUCH more like a fourth season of TOS. It would all depend on how well done is was, but I’d sure be open to seeing what a dedicated team might come up with.
I believe it would be a reasonable compromise to remaster from film just the parts of the show that had no CGI visual effects, and work out CGI scenes with this smart upscaling technique. It wouldn’t be as shiny as TNG, but wouldn’t be much behind Enterprise’s own HD masters, which have a lot of CGI effects in SD in the first two seasons. That could prove to be cost-effective.
I believe it would be a reasonable compromise to remaster from film just the parts of the show that had no CGI visual effects.
The problem with that, is that Trek shows are incredibly VFX heavy. Many scenes have some form of post-produced effect in them. So where do you draw the line?
Any time someone comes on the view screen that would need to be recomposited or it would need to drop to standard def upscale for those angles that we see the screen.
There are also many things people often just take for granted were done with early tools that “digitally painted” in effects like phaser fire, transporter effects, some of the more involved display graphics were not done on set, etc. And of course Odo morphing was an early example of CGI, based on the success of Terminator 2. By the rule that only scenes without visual effects are re-scanned, there would be many scenes where it would dropping to SD every few seconds.
Here’s the thing, if you’re spending the money to go get the 35mm film reels out of storage, and have them sorted and cataloged and scanned in. That’s a huge labor cost already. Why would they spend all that money, and then just stop? The answer is they wouldn’t, if they’re spending the money, they either need to budget for the all-in cost of remastering or they won’t bother.
but wouldn’t be much behind Enterprise’s own HD masters, which have a lot of CGI effects in SD in the first two seasons.
That’s a different scenario. Enterprise was actually post-produced in HD. Some of the space shots and some of the greenscreen fully digital backgrounds were rendered lower resolution for time and money reasons, but they were then imported into the HD video editing workflow along with the live action footage which was filmed on 35mm and then scanned in at HD resolution. The other visual effects that people tend to take for granted, like phaser fire, were always part of the HD workflow, you can tell it was done natively in HD because scenes with phaser fire or other such effects don’t drop to SD. Regardless of what was done to the space shots, the final output format of Enterprise was HD.
Yeah, I am aware of that. And it is just a matter of drawing a line, taking into account the available budget, and get down to it. I personally would pursue a “when all elements of any given shot are available, go for the remaster; when there are video effects/CGI, go with upscaling”. That would ruin every Odo morph, and every phaser fired, but would bring all model work, plus screen communications and effects-free shots to all its HD glory. It’s a tough call. TNG accepted a few seconds worth of upscaling for the whole series. We’re talking about a few hours worth of upscaling. But at least there would be *most* true HD, at a reasonable cost.
The alternative — no HD whatsoever — sounds worst.
Of course, given no money constraints, a full remaster should be the way to go. But I’m affraid there’s no faith in this, and I can’t really blame the powers-at-be. Home video is dead, and streaming is not the best place for quality purists…
This ‘home video is dead’ thing sure seems massively premature to me, and a huge step in the wrong direction. Perhaps it might turn out to be more of a generational/age issue than a lasting one. I sure hope it reverses course without having to go through a 30+ year exile like vinyl.
Why? Because when everybody gets their entertainment from a single source, you can bet it all that source is going to start tampering with the content — in small ways at first, but even so — to suit its own ends. We get enough of that with time-compression during broadcast TV already. I want to be able to put TWILIGHT’S LAST GLEAMING on and watch it as it was made, not with a ‘happy ending’ or with the cussing deleted, same as I don’t want to see (and longtime posters have heard me say this before) MOBY DICK where Ahab suddenly grows a bionic leg and uses it to kick the whale to death. If the current US government was controlling media distribution and access, can you see ALL THE PRESIDENT’S MEN being streamed at all? And how big a leap is it from that to not showing or modifying ARENA, where our guys admit, ‘we could be in the wrong.’
Vinyl’s ressurrection is overrated. It is a niche product, at best.
I personally am an enthusiast of physical media, and collected all TNG season boxsets on BD, but the trend is clear.
Maybe in 30 years they will make a comeback. Maybe it happens even sooner, thanks to the streaming wars and the fact that you will have to subscribe 10+ services to get all the catalogue titles you want. But right now it doesn’t look good. Unfortunately.
Salvador Nogueira,
One reason for technology resurgences over time is patents don’t have infinite life spans, in 30 years there’ll be no licensing fees for manufacturing blu-rays or making something that can play them. While Disneyfied copyrights will likely remain unexpired there’ll be a lot in the Public Domain – not to mention already pressed discs – that can benefit from this.
There’s been a growing backlash when things are removed from streaming, and that many people are learning the hard way that the digital version of movies and shows that they’ve “bought” but don’t actually have physically (either as a downloaded file or as a physical media), end up disappearing. Whenever something is bought digitally, it can disappear. They’re just renting it for as long as the service has it available. But these people are a tiny minority.
I think that people largely don’t care if they can’t find particular things available to streaming, they’ll just watch something else. This isn’t a youth issue, I know plenty of Gen-Xers who are just as bad, if not worse, than what people would think is just an issue with Millennial or whatever the generation after Millennials is called.
I don’t buy a whole lot of things physically anymore, but I do focus on high quality, worthwhile releases that can fit in with my “collection.” Things like Star Trek Remastered and the recent classic Doctor Who Blu Ray releases are among them.
Long story short: People will just continue to complain that they can’t get X movie or Y show on Netflix or Prime, and then, after a few moments, find something else to watch.
More and more, I find myself relying heavily on my very large collection of DVD and Blu-Ray rather than streaming Amazon and Netflix. The thing about making a decision about what to watch as opposed to just defaulting to what is available is an important one. I remember this awesome 2-part interview Nick Meyer did back in the 80s for ENTERPRISE INCIDENTS magazine, and he stresses that it makes a huge difference when you choose to watch something — get up, rent the movie, or take the box off the shelf, and put it in — as opposed to just sitting there and accepting what is offered.
My analogy would be to reading … I am a voracious reader, but it is very rare that I ever just grab something at random, like a mag in a doctor’s office. I have something I want to read or reread. then make sure to have it on hand. I really don’t want to fill up brainspace with US WEEKLY!
I do have to say that streaming opens me up to revisiting old bad movies that I am sometimes in the mood for and would never choose to own. Lately I’ve been itching to see a couple of the later even more heinous Irwin Allen movies, but found that except for their trailers, they aren’t available to stream. But just seeing the trailers was probably enough rather than investing the full 2hours. But again this viewing is a matter of choice rather than just accepting what is offered — if the need to see WHEN TIME RAN OUT got bad enough, I’d find a way to acquire it (and feel bad for spending the $ afterward.)
You nailed it with one word ”COST”If they went that far as you say why not just do it right like next gen. People just don’t get it cause they know noting about the time and the cost to do that. I’m with you they won’t do it. Didn’t anyone learn from DS9 profit, profit profit. The amount of Fans that would even buy a DS9 blu-ray is small, Star Trek as much as we love it doesn’t have that big of a fan base as we think and when a new series comes out most of the comments I see here is bashing the new series. That bitching about Discovery sends a message to the producers of the show and most of all CBS is that why bother making it better ,they bitch anyways. I’m sure when the PICARD series comes out fans will find something wrong with that show also. If you wonder as fans why Trek doesn’t get the love it deserves, you fans are part of the problem. That’s called voting against your own interests. Shut up and enjoy what your getting. You sound like entitled little bitches.
Matt Wright,
Wait, I get that you are saying that they aren’t going to respond to market demands to produce a finished product, but are you saying that as a matter of protecting their archived filmed assets from unanticipated disasters they DON’T rescan their archived film reels every few decades or so when a leap like 8K proves feasible for backup digital archiving of the asset?
For example, Paramount does indeed periodically digitally preserve/archive their film catalogs, but they don’t see the benefit of preparing it for a consumer release.
CBS is different matter, I don’t know much about what they do, but I do know all the 35mm film for the Trek shows is securely stored/preserved in a former salt mine (along with other Paramount assets since they’re former corporate siblings).
Interesting, I’m wondering when will get the first 6 movies and 3 next gen movies in 4K without the DNR mess they did on blu-ray. I thought Trek was their cash cow.
I thought Trek was their cash cow.
Paramount says that, but they’re not willing to spend money on the back-catalog.
Paramount was in a really bad place financially for a good handful of years there, so they became very adverse to doing anything outside of surefire wins for their bottom line.
The trend in home video among pretty much all studios now is to wait until a new entry in the franchise comes out in theaters and then they feel they can get in on the hype and release new versions of the catalog titles. Examples: Before Mission: Impossible – Fallout came out Paramount issued new UHD BD versions of every M:I movie. With the newest Avengers movie now only a month away, Disney has just released the first Captain America on UHD BD. Similarly, when Avengers: Infinity War came to home video last summer Disney also issued the first two Avengers movies on UHD BD with it.
Of course there’s no new Star Trek movie in sight now, so who knows if/when they’ll do it?
I don’t know Matt. You would think they would with the video age here forever.I remember reading here when they were working on Next gen for blu-ray and the amount of work just to get the film elements up to par. color correcting etc.I saw pictures of film in storage cases but still they had to do all that work.It seems to me Hollywood is always a tick behind the new tech.I wonder knowing 8K is just down the road around the corner are they even thinking about backup. Some movies are shot in 4K however many are still shot in 2.5k then upscaled to 4K. I mean I don’t think there ready for 8K or beyond.
And you would pay how much for this wouldn’t be as shiny as TNG stuff.Nope if I have to pay close to 100 bucks a season at release time I want it done right.
Thomas P Vinelli,
Unfortunately, I’m old enough to remember when unauthorized 16mm film dupes of a syndicated reel distribution of a first series episode went for twice that, so I wouldn’t balk at that price. But I am a bit of a film archivist purist, so if they cut corners on that also being available to blow the wad on the new VFX, the value to me decreases.
This looked surprisingly good to me on my 4K monitor.
This exercise was to see what would happen if he applied the algorithms.
Thanks for sharing these, it was fun to see them.
I’d love to see a remaster of the series, just for streaming sake.
I’m not sure if there would be a return on investment for CBS.
For those of us who love the show we would probably pay to buy BluRay or give it a binge on our streaming service. I doubt it would draw in many new fans due to its 4X3 aspect ratio.
Watching anything on a 4X3 aspect ratio takes me out of the story.
I suspect new viewers would have the same response.
Awww, I’m in love with these HD clips. <3<3<3
C'mon CBS. DS9 and Voyager deserve the same love as TNG.
In your dreams ,sad to say , won’t happen.Sales were really bad with Nex gen, TOS was better.But Next Gen was the killer. If DS9 was to be in HD it would have happened already. Money talks and everybody else walks The cost of doing these series ,7 seasons about 22, 23 episodes nah ain’t going happen.
Matt Wright,
Does anyone know how DS9, back in the day, was converted to PAL for syndication to the UK and other PAL markets? PAL has better color and higher resolution than NTSC.
As far as I know, but it’s hardly definitive, it was an NTSC to PAL video conversion. The show was finished on professional grade video tape, so without remastering it from scratch, it is forever locked to NTSC.
According to wikipedia, the pro D1 video tape decks recorded in a way that output could be compatible with either NTSC or PAL, but I don’t think it really offered increased image quality to do so.
Conversion was NTSC to PAL. It made the episodes even run faster (with slighted distorted sound), thanks to converting 29.97 frames down to 25 frames (PAL’s standard).
This was a big question 30 years back, because I remember the CINEFEX article on TNG, and how Peter Lauritson realized going in that they wouldn’t be able to release the show in Euro markets or anyplace where TV standard were higher than domestic SD, so I wondered how the show did with PAL. Even back then it seemed a massively shortsighted and wrongheaded solution, but given the expense of post and how the show was being done, it was actually heralded and imitated — which may help explain why a lot of other stuff from late 80s and 90s looks like utter crap.
Any conversion would have had some generational loss, plus the quality loss when you’re going from an inferior format to a larger one … it’s one reason why I always made a point of avoiding movies shot in 35mm that were blown up to be shown in 70mm at big movie theaters in the 80s and 90s, you were just paying to see four times as much grain and a lot less of the original image quality (TUC was a prime example of this, because being a Super 35 show, the entire film was essentially an optical blowup already, BEFORE the 70mm version.) I remember a dissolve in SILVERADO where the image fell apart massively even in the 35mm version, so I’m thinking it must have looked like the big image dropoff redolent of a 50s scifi flick when screened in 70mm.
kmart,
That was my thinking too, i.e. they to do something to make these shows competitive in terms of image quality in PAL markets.
I thought I remembered the D1 could output PAL and just assumed that’s what they went with, but a lot of articles seemed to indicate that a lot of the studios just upconverted the NTSC which just seemed crazy to me.
I called this out a while ago that this will be the way how DS9 and Voyager get remastered in a few years! And no, it would not be the SAME as a proper re-compositing, but it could be just as GOOD! Have you guys seen how it is now possible to turn summer into winter and night into day in videos? And this is only the beginning! Especially if fed with large numbers of high-quality production stills and photos of DS9 (or just googling them on its own), a ML algorithm could deliver a very convincing result and fill in the blanks on its own, in a few years of time.
As Deckard said, ENHANCE!
Enhance has to be more than just turning a knob. You have to have more info to squeeze out of the image. The picture Deckard was enhancing had some kind of holographic data in it, because you could track as well as zoom, changing perspective to see things that wouldn’t be visible in a straight capture, so that is an excellent example of how the original capture sets the limits on what is possible later with the image (unless you’re tweaking to create new data, which would be cheating and/or massively expensive on a global scale.)
When you say, change night into day, all I’m remembering is how bad it looked when the Veridian star went out and they just darkened the day plate of Picard’s closeup … there are a lot of values to address when transforming an image or stripping it into a different environment, which is why so much of what was well-shot originally in A WRINKLE IN TIME turned to crap when they comped elements shot in one lighting situation into vastly different environments, against the intent when the shooting was first done.
I think one point to remember is that this is not a forensic investigation and as such the Enhance analogy shouldn’t be taken at face value. What I was comparing is the idea of lossless zooming, but how we go about it is different. We do not need to “squeeze out more information” from the original but instead can add new information from other sources with real HD material – DS9 high rez shots, TNG’s Birthright and the coming DS9 documentary. What GANs do is “hallucinate” details into images based upon the images they have been trained on.
As for looking around the corners, have you seen the recent examples of reconstructing the original image based on shadows on a wall?
I did some interviews a few years back about the plenoptic capture processes, and even that sounds a bit too removed from conventional photography and cinematography to suit me, so these processes you cite, while of interest by themselves, sound … I dunno, inappropriate or unfair to apply to finished works. Then again, I’m no fan of TOS-remastered either, so in general I’m not in favor of anybody tampering with a finished work.
The one exception — and I said this back in the 90s — was I thought recompositing original FX elements using newer techniques would be okay, because you’re still limiting yourself to the tools of the times. The 21st century final cut of BLADE RUNNER had all the matte composite shots redone digitally using original elements (which the director himself had paid to have preserved — a far cry from the weird situation with TMP effects elements, where parties offer up radically different accounts about them), and that was a pretty transparent process that didn’t damage the look or change it in any significant way (the original work was so good that the recomps probably weren’t even necessary, something I don’t say about almost any other film outside of 2001.)
Oh, so we are arguing the ethics. Granted, AI upscaling is not strictly only using the original work, but as described, using HD background material from the same show, isn’t it still much more faithful and respectful to the original than recomposition and even CGI replacements a la TMP or TOS could ever be? In fact TMP is a good example as it changed the original considerably AND the new effects itself are outdated already again. AI upscaling avoids both of these issues – the content stays exactly the same, and similiar to vectorization, the end result can be upscaled to any target, limited only by computer power.
Hmm. Well the TMP upgrade was out of date from the first render IMO. To have the necessary resolution, they’d have been better off animating high quality still photographs of the ship (a la 2001 for many shots) … the large-format stills shot at the end of TMP are amazing in their detail (I used to have a 16×20 photo that seemed to have more detail than I ever actually saw in the film), and if you shot the results in 35mm, you’d have been good to go for the next couple or three decades at least.
So if we proceed with the smart tech solution, do we get the cinematographer and VFX supe in to sign off on the look as being in keeping with their intentions? Or is it a producer’s call? And what sorts of rights are there in terms of the original work, and royalties? If it is a new image derived from the original work, are the royalties the same, or reduced?
I guess I’m seeing the glass as half-empty here, but that’s my kneejerk takeaway.
While this isn’t quite up to the level needed for an effects heavy scifi show like DS9 or VOY, this would be a great way to upres shows that otherwise wouldn’t or couldn’t be remastered to HD. My girlfriend was a big fan of Dr. Quinn Medicine Woman and Xena…maybe shows like that could be upgraded with this technique.
But the question is how much will they charge a season and are you willing to pay top dollar? Because they will try to suck your wallet dry as much as they can get away with. Since this product would be upscaled and the results not quite HD nah, not interested. There still asking top dollar for DVD seasons of DS9 480p
1) it all looks exactly the same as before. I lol at anyone delusional enough to squint their eyes and tell themselves that there is a difference.
2) *every* digital tv rescales everything that arrives at any non-native resolution, and in real time.
What quality did you view it at? It looks the same at 480 on YouTube, but at 1080, there was a subtle difference in most of the scenes. Not really enough to go ga-ga over, but certainly not “delusional”.
It really only seemed to benefit the space shots, where extra definition was given to the ships. The closeups really didn’t provide any real room for improvement.
I’ve said elsewhere in this thread that this is a half-measure. It’s probably not worth the time investing it over a full remastering from the source elements, especially when we can see the night-and-day differences in the remastered shots done for the DS9 documentary.
I would like to see CBS get together with this guy to obtain his AI process and see if they can apply it to the original source material. Then we might see dramatic improvements… possibly even 4K without a lot of hands on effort.
They don’t need to ask — he already told us. He just plugged each frame (as a batch) into AI Gigapixel, at pretty much default settings.
I don’t know. It gets kind of a CGI look. Not sure if I like that.
I honestly don’t see a big difference and I have a 1080p monitor.To truly get a HD master you would have to do a total restore Like was done with TNG ,not a remaster which are 2 different things.And CBS will never do that. DS9,Voyager, 480p is a far as it will ever go. Even Enterprise was done horrible and that was in HD.
How about one for Star Trek: The Motion Picture 2001 version?
Way, way overdue
Many of us would be supportive of that. Just a reminder: Paramount owns the movies, and CBS the TV properties. Paramount is terrible with their back-catalog of films, not just Trek. For example, they’re sitting a beautiful 4k restoration of The Godfather and can’t be bothered to release it. For Trek they have a TWOK 4k version that was created in 2016, last year they very quietly released only as a digital version (iTunes, Google Play, etc.), but they apparently see no reason to release it on disc.
In comparison CBS has done amazing things, as they’ve spent the money to remaster TOS and TNG which is 10 seasons of TV between them.
I agree Paramount sucks and according to the filmmakers of the past Paramount is cheap. I wonder if the real answer is the head of Paramount isn’t a big Trek fan. In the past those first 6 movies were made because the head of paramount was a Trek fan. Just speculation.
It isn’t just TREK with Paramount when it comes to not delivering. They won’t release THE PARALLAX VIEW or LOOKING FOR MR GOODBAR on BluRay either, to name just two of my alltime faves. Didn’t know about that 4K GODFATHER, that is truly a shame.
I would greatly prefer a 4K theatrical TMP rather than the so-called director version, whether it was cleaned up or not, just because I know how good the Enterprise miniature looked in the theater and that I haven’t gotten that on homevid yet with the existing blu-, plus I think the ‘improvements’ on the director cut dvd are mostly not improvements and in some cases, like the sound mix, they trash parts of the movie for me.
IMO, the ideal release of TMP is a version with seamless branching where you can choose DE or Theatrical.
I’d be in favor of an edit-your-own version, that way I can cut out that last shot of the astronaut fleeing ep IX, where it looks like somebody threw one of my old Major Matt Mason toys at the camera. Of course I’d also need the option to replace all the Vulcan matte paintings in both versions with the original one that was tossed when Trumbull revised things, because that one actually looked good. And I’d put back in the very nice San Fran shot (side wide view of tram flying towards Starfleet, which I’ve seen in still form but didn’t get finished in time to go into the theatrical release. And re-time the color on all San Fran shots to evening warm, instead of the terrible blue that was inflicted on it after the shots were done, which is why the paintings in that scene look so dead.
Y’know, the more I think about it, the more I want to MAKE my own version of an idealized TMP … one where Sulu is really having to work hard to keep pace inside the cloud with a 78 kilometer vessel as they both travel at warp 7 plus …
I didn’t know they had gone ahead and released the 4k remaster of TWOK for digital release. Bastards!
It looks like video not film. That is the problem with upscaling it does not create a sharper more defined image.
I also think the different was minimal. I see it in contrast, some level of dark or light tones. But the problem with DS9 was the fuzzy look, the tarnished or bleeding colors. CGI or not , it lacks the clean and crisp look of HD. I don’t think an AI can compensate for that. I am surprised that it’s less complicated to process an old filmed video, then a digital video like DS9. I would have though the filters would be easier to control and applied.
Once you put a filter on something shot live it is hard to remove that effect if it isn’t consistent. RED PLANET was a movie shot with a red filter, and then the vfx companies (I think 11 of them in the end) had to wait while one vendor came up with a way to extract that look so that a uniform red could be applied in post for all the shots. So on-set filtration is a problem regardless of there being a film camera or not. And DS9’s prob is that it was shot in glorious 35mm but finished in SD, so whatever gorgeousness was there in the image just goes to mush right out of the gate. DS9 for me is almost like radio, because sometimes the looks is so bad on DVD and streaming that I close my eyes or try to sit further away to not be distracted by how terrible it looks. And that’s not good, because it undercuts all the great early VFX work on the ships and the often awesome stuntwork in the fights. I’d probably pay twenty bucks just to see WAY OF THE WARRIOR as intended (and since that one would only have phaser blasts to recreate, because all of the ships were still practical and shot on 35mm, it would be a helluva test bed.)
I really don’t see much of a difference…but that’s just me.
You can achieve the same effect by simply increasing sharpness and contrast.
https://www.youtube.com/watch?v=3RRGcsqhiJ4