Bruce's Thoughts on PS3 Horizontal Scaling in SDK

Some of you might remember I pinged Bruce Dawson (one of our senior software design engineers here) earlier with some questions around 1080i and 1080p. At the time I was trying to understand where Sony was coming from with all their "TrueHD" talk, especially as the 360 had just enabled 1080p output as well.

Anyway, last week we heard the news that the PS3 had fixed scaling on 480i PS2 titles running in backwards compatibility mode. This is a good thing. But we also heard that the PS3 SDK had been updated to allow PS3 titles to scale their content, but only horizontally. This is a weird thing.

I couldn't wrap my head around all the technical implications here, so I asked Bruce for his thoughts. Since there's nothing confidential here, I thought I'd share what he sent me. He knows his stuff, so if you find something you strongly disagree with or just want to ask a question on, add a comment to the discussion. I'll ask Bruce to swing by and answer as he has the time. Bruce's thoughts follow:

I see no sign that the PS3 contains a chip that can do vertical scaling, and this new feature (horizontal scaling) is a poor substitute for a true hardware scaler. It is a step forward for owners of 1080i only HDTVs, once PS3 games support it, but it is nowhere near as good as the Xbox 360’s scaler.

This recent announcement is just for horizontal scaling, and horizontal scaling is easy. To do high quality horizontal scaling you just need to buffer up a few pixels and intelligently average between them.

Cheap horizontal scaling is even easier: you just send pixels to the video output a bit slower (or send pixels at the same rate, but read them from memory slower). It’s the sort of thing that consoles have always been able to do. This new horizontal scaling feature just sounds like they are adjusting the video output rate.

Vertical scaling, on the other hand, is much harder. You need to be able to buffer up (or sample from) two or more lines of data, and then intelligently average between them. For high quality scaling you want to be sampling from a half-dozen lines or more. The Xbox 360 can do this. I don’t know whether the PS3 can do this, but if it could I think we would have seen it by now.

This new feature means that games that have previously only supported 720p can now, sort of, be modified to support 1080. When these games detect a display that can’t support 720p they can switch to using a 960x1080 buffer. This is only 12.5% more pixels than 1280x720 so the increase in fill rate and memory consumption should be manageable. Then they can tell the PS3 to stretch this buffer to 1920x1080 at display time and voila, 1080 support.

Except, it will be pretty weak 1080 support with an odd and substandard result. The horizontal resolution will be worse than with 720p (960 across instead of 1280), and the images will be twice as blurry horizontally as vertically. 960x1080 is going to look worse than 1280x720 (although it’s certainly an improvement over having to drop back to 480i).

19 comment(s)

 avatar
Rask wrote on January 23, 2008

I had figured that Sony didn't somehow manage to hide a hardware scaler from the general public and that there had to be some sort of trick to it.

If Bruce is right, then that may explain it.  Regardless, for bringing this tidbit to light, you may get some flaming from the PS3 fanboys =).

Regardless, at least it is some sort of fix for the 480p/1080i owners which is a step in the right direction.

 avatar
wrote on January 23, 2008

Re: "Regardless, at least it is some sort of fix for the 480p/1080i owners which is a step in the right direction."

Yep, totally agree. And the backcompat fix is all goodness as well. All moves in the right direction!

 avatar
Eastbeast314 wrote on January 23, 2008

Very interesting post - thanks!

 avatar
nopants727 wrote on January 23, 2008

Hey Ozy, I have some questions for Bruce if you don't mind.  The x res of 960 is 75% of 1280.  So, does that mean every 4th pixel from a line is dropped and then that line is doubled (i.e. 1010 becomes 11001100)?  Although, I'm guessing some averaging is done to smooth that over.  Also, if there is no vertical scaling taking place, how do you get the additional 360 lines to make the image 1080 in the y?  I don't know much about digital video, but is each line on the screen outputted serially?

 avatar
J.Goodwin wrote on January 23, 2008

His math doesn't make any sense to me either.

 avatar
Anon Math Major wrote on January 23, 2008

It's not that difficult, people. If you have a 960x1080 framebuffer, then to output 1920x1080 you just send out every pixel twice (assuming you output line-by-line), like so:

AABBCCDD

EEFFGGHH

Voila! Your horizontal resolution has doubled. Or to a bit more fancy, you buffer every two pixels and output the average between them:

A(A+B/2)B(B+C/2)C...

The point here is that horizontal scaling is easy because data is almost always output row by row. Vertical scaling is hard because you'd have to buffer entire rows of data and THEN double them or average between then.

The math is easy, too:

1280x720 = 921,600 pixels

1920x1080 = 2,073,600 pixels (125% more than 720p)

960x1080 = 1,036,800 pixels (12.5% more than 720p)

Got it now?

 avatar
logikil wrote on January 23, 2008

Math Major:

Thanks for clearing that up. After seeing what you did there I had a Duh moment. Much appreciated.

 avatar
nopants727 wrote on January 23, 2008

Math Major-

Thanks for that explaination, as it is very easy to understand.  My questions still stand though, as you repeated what I stated, only much better (I suck at asking questions).  If there is no vertical scaling, where are the additional 360 rows coming from?  Is this done by just repeating every odd row?

720 rows:

0

1

0

1

1080 rows:

0

1

1

0

1

1

Is this similar to what an interlaced to progressive line doubler does?  I also don't quite see how a slower bit rate can produce visual scaling.  Is it some sort of Nyquist aliasing effect?

 avatar
nopants727 wrote on January 23, 2008

Besides, I only have a lowly Math minor. ;)

 avatar
MH4 wrote on January 23, 2008

The extra 360 rows come from the game. The game has to recognise the problem and actually render at 960x1080. For instance, Resistance will never be able to take advantage of this without a game update.

 avatar
Anil wrote on January 23, 2008

I still don't understand how adding more pixels horizontally can solve the problem? its like stretching an image just horizontally. you will end of seeing some fat objects on your screen. Someone correct me please?

 avatar
SeanB wrote on January 23, 2008

Anil- The aspect ratio will be correct since the game will know that each pixel is going to be repeated horizontally. So the game will draw everything in its framebuffer half the width that it is supposed to be. When it gets scaled to the full resolution, things will be the normal width. I imagine this will blur things a fair amount, however (imagine 540p to 1080p, not far off from stretching 480p to 1080p, but horizontally instead of vertically).

 avatar
Anon Math Major wrote on January 23, 2008

Re: the 360's scaling from 720p => 1080p, it definitely does NOT double the line and drop every fourth pixel, since clearly it'd be losing data that way. Not to mention that it would generally look like crap. The scaler does some sort of interpolation, which is fancy way of saying that it guesses to values of the other pixels based on the pixels it knows. It probably does linear interpolation (which is basically like a weighted average), but there are other interpolations, too (such as polynomial). I don't think those would be very applicable to scaling an image, though.

 avatar
MikeM wrote on January 23, 2008

(WTF are these guys talking about)???

...

...

TECMO FOOTBALL RULZ!!!

 avatar
J.Goodwin wrote on January 23, 2008

I get the horizontal part (that's easy, since televisions have always been about scanlines as opposed to horizontal pixels).  The vertical part is where I was getting nothing.  It makes sense that the game would need to render appropriately though.

Which, eh, is kind of pointless, since basically you end up needing to do the extra work either way.  It's all a fancy way of saying that someone has to program the game to do what it was supposed to do in the first place, whereas on the 360, you have a piece of hardware capable of doing all that work automatically (which actually seems to be very similar, like a great many things about the 360, to the way that Dreamcast worked).

 avatar
Techni wrote on January 23, 2008

Minor problem Ozy, since the scaling requires two framebuffers (the second for the upscaled image) that means it's not relying on the scaling method you suggest (holding pixels longer ont he output to increase width) and that it is being scaled internally.

Also, you were claiming PS3 didn't have a scaler at all, and were wrong about that. You're probably wrong about it not having vertical scaling abilities as well.

Oh and 360 has the same blurriness.

Please stick to commenting on your own system.

You have enough problems on the 360 to deal with without looking at PS3s.

 avatar
Anonim wrote on January 23, 2008

<b>

PS3 DOESN'T HAVE HARDWARE SCALLER

-it's a TRICK to do horizontal *stretch* ONLY

-REQUIRES ANAMORPHIC [squashed] picture to work [like this:]

http://i138.photobucket.com/albums/q262/anonim1979/oblivionanamorphicmaly2razem.jpg

http://en.wikipedia.org/wiki/Anamorphic

-only 3 resolution supported 960x1080 1440x1080 1600x1080

-at least 12.5% more costly than 720p

-REQUIES GAME ENGINES REWRITE to work with older and new titles

-ENGINE MUST have 2 DIFFERENT modes suported - for anamorphic output AND normal output [rest of resolutions]

-it works by doubling [for 960]/puting 1 extra pixel for every 3 pixels [1440]/puting 1 extra pixel for every 4 pixels [1600]

-WON'T WORK with PS1 / PS2 games

</b>

It does NOT upscale to 1080 - frame is already in 1080.

for example 960x1080

sending lines 1,2,3,4,itd from 1080

abcdefghijk....

abcdefghijk....

abcdefghijk....

abcdefghijk....

it simply addas pixels:

aabbccddeeffgghhiijjkk....

aabbccddeeffgghhiijjkk....

aabbccddeeffgghhiijjkk....

aabbccddeeffgghhiijjkk....

X360 scaler works in all directions, it's similar to analog knobs on CRT monitor stretching picture.

PS3 uses TRICK - uses probably changed timings in output - puting extra pixels

like somebody said it's a trick similar to that used on PS2:

[quote]

Uh, isn't this the same thing that the PS2 did when it went into "16:9" mode in 480p? They didn't actually increase the output resolution... they just, umm, squished the image so that it looked correct on a 16:9 display. :|

[/quote]

 avatar
homeboy wrote on January 23, 2008

The 360 ANA scaler is even cheaper in design because it's just an upscale chip in analog signal just likes what Anonim described.

 avatar
ni9ht 5ta1k3r wrote on January 23, 2008

I also saw the original article and thought that it won't scale blu-ray movies because this only effects games.