[devel] loading non-premultiplied alpha images. help needed

cocos2d for iPhone » Programming » cocos2d internals

[devel] loading non-premultiplied alpha images. help needed

(20 posts) (4 voices)

Started 2 years ago by riq
Latest reply from riq

Tags:

riq
Key Master

trunk (SVN r1829) contains the patch to load non-premultiplied images.
The patch follows the Apple's recommended way:
https://devforums.apple.com/thread/3877?tstart=0

But unfortunately (perhaps I'm doing something wrong) it is still loading premultiplied images.

The code that I'm using to load the non-premultiplied images is here (CCTexture2D.m#initRGBATextureWithImage):
http://code.google.com/p/cocos2d-iphone/source/browse/trunk/cocos2d/CCTexture2D.m?spec=svn1829&r=1829#342

And as you can see from this discussion, I tried many approaches:
http://code.google.com/p/cocos2d-iphone/issues/detail?id=697

So, my questions is: Do you know how to load non-premultiplied images on the device without resorting to external libs like libpng ?

Posted 2 years ago #
CJ
Moderator

Riq, I've been closely following the progression of this whole premultiplied alpha thing and textures. I even ditched cocos2d's png loading for libpng entirely for about 6 months until the issue appeared to be resolved in cocos2d.

Now I'm 99% sure after reading everything that it is actually desirable for the device in terms of performance to use pre-multiplied alpha images and setting the openGL blend mode accordingly.

So I think the best approach for Cocos2D would be to accept images in this format gracefully and set the blending modes accordingly as it should be the best performing version of the image data.

...

So if I'm off about the above, can you explain what the benefit of attempting to circumvent the pre-multiplication of alpha?

Posted 2 years ago #
riq
Key Master
@CJ: For example, you can't have opacity and color at the same time.
Suppose you want to modify the color of an sprite:
```
sprite.color = RED;
```
and now you want to do a fade out:
```
[sprite runAction:fade_out];
```
You can't have both: either you have opacity or color, but not both at the same time.
Of course you could do it if you change the blending mode, but that wouldn't render the image as you meant to be.

Regarding the performance, why do you think it is faster to use pre-multiplied images ?
Although I've not done any performance test with the glBlend, I would say that all the blending functions should have more or less the same performance .
Posted 2 years ago #
CJ
Moderator

@riq: I actually posted code to deal with the case of opacity + color + premultiplied textures:
http://code.google.com/p/cocos2d-iphone/issues/detail?id=668
with forum discussion here:
http://www.cocos2d-iphone.org/forum/topic/3888

Well regarding the performance, I'm still digging on the net to find the reference (just re-read the OpenGL performance article in the iPhone docs), but I cannot find it at the moment.

Posted 2 years ago #
riq
Key Master

@CJ: Thanks. I'll take a look at the patch.

As far as I remember, the iPhone docs recommends using "alpha blending" instead of "alpha testing", but says nothing about specific blending functions.

So, if the performance is the same, I would like to have non-premultiplied images by default in cocos2d.

Posted 2 years ago #
Steve Oldmeadow
Administrator

My understanding was if you let XCode "compress" your pngs it not only pre-multiplies the alpha but converts RGB to the iphone's native BGR format - all in the name of performance. However, it may not make a huge difference in practice - it would be interesting to get some performance comparisons.

Posted 2 years ago #
CJ
Moderator

@steve yes I think I read that as well. Do you or anyone know if OpenGL takes the images in BGR format, or does the loading code end up putting things back into RGB order?

If it stays in BGR format all the way through to OpenGL then I would guess that it's the best performing version, but of course only tests will tell.

Posted 2 years ago #
riq
Key Master

I think iPhone supports both BGR and RGB textures, but I think they have the same render speed.
The Xcode "compress PNG" feature makes the loading time a bit faster, but as far as I know the render time is the same.

Posted 2 years ago #
CJ
Moderator

More food for thought:
http://osdir.com/ml/quartz-dev/2009-05/msg00030.html
(follow the thread to the end.)

Posted 2 years ago #
Karl
Member

Very interesting!

On a side note, when I started loading pixel data directly into the texture buffer rather than via a PNG, I noticed that in the 16-bit modes, the pixel data must be in little endian format.

So the pixel data looks like this for RGBA4444:

Offset + 0: hi = B, lo = A
Offset + 1: hi = R, lo = G

And RGB565:

Offset + 0: lower 3 bits of G, followed by 5 bits of B
Offset + 1: 5 bits of R, followed by upper 3 bits of G

And yet in 32-bit RGBA8888 mode, it's like this:

Offset + 0: R
Offset + 1: G
Offset + 2: B
Offset + 3: A

Posted 2 years ago #
Karl
Member

Another thing: Any time I attempted to load non-premultiplied data directly into the gl texture (by passing it to Texture2d.initWithData), it freaked out and gave me garbage on the screen.

Posted 2 years ago #
riq
Key Master

Another article: benefits of premultiplied alpha
http://home.comcast.net/~tom_forsyth/blog.wiki.html#%5B%5BPremultiplied%20alpha%5D%5D

It seems that r1833 loads OK non-premultiplied BMP images, but not PNG images.

I'll try with CJ patch. If it works as expected, then v0.99.1 will continue to use premultiplied images by default.

Anyway, I would like to know if it is possible to get non-premultiplied PNG images on the device using UIImage and CG. In theory it seems it is possible, but in practice it seems it is not :(

Posted 2 years ago #
Karl
Member
Are you skipping PNG preprocessing in your project?

According to http://plaidworld.com/iphonefaq.txt

You need to add this to any target that you want to skip PNG processing on:
```
IPHONE_OPTIMIZE_OPTIONS=-skip-PNGs
```
But you also have to be careful since an unprocessed Icon.png and Default.png will not display correctly.
Posted 2 years ago #
riq
Key Master

@Karl: yes, I'm doing it, but I think that IPHONE_OPTIMIZE_OPTIONS is deprecated.
There is a new option called: "Compress PNG files" which should be disabled.
I'm using both of them, and I know that the PNG images are not compressed because I'm loading them with libpng, and libpng renders them OK.

Posted 2 years ago #
CJ
Moderator

Here's an article on Particle Systems:
http://realtimecollisiondetection.net/blog/?p=91

The interesting (relevant) part is just a single statement:
Using premultiplied alpha (which is associative) we can blend particles front-to-back instead of the normal back-to-front ordering.

Unfortunately he doesn't explain it much further. But I know iPhone uses the TBDR rendering scheme and in the iPhone OpenGL performance recommendations it explicitly says that you don't need to sort back to front when drawing. And you should rather sort by gl state and alpha vs non blended objects.

Posted 2 years ago #
CJ
Moderator

@riq
Just read the article here:
http://home.comcast.net/~tom_forsyth/blog.wiki.html#Premultiplied%20alpha

Wow, that is some good info. Great find :)

Posted 2 years ago #
riq
Key Master

@CJ: fixed issue #668. SVN r1835. thanks. premultiplied images are the default ones.. Actually the "non-premultiplied" code is commented. Also, it seems that the non-premultiplied code is not working because of a bug in the iPhone SDK. Opened ticket #7701261. Even though apple fixes it, premultiplied images will be the default ones.

Posted 2 years ago #
CJ
Moderator

@riq Excellent :)
Great work finding all of the information about this. Even though I supplied the patch, I didn't fully understand the whole pre-multiplication issue until after the article you posted here :)

Cocos2D is the best!

Posted 2 years ago #
CJ
Moderator

@riq I was going through Apple's sample code today, and I came across a Texture loading example that manages to get the texture in w/o alpha pre-multiplied. Now I know we all agree that we won't be using this as default, but for the sake of completeness, here's the link (dev account required I think): http://developer.apple.com/iphone/library/samplecode/GLImageProcessing/listing12.html

Posted 2 years ago #
riq
Key Master

@CJ: thanks. That's the code that I've been using. If you try it, you'll notice that it loads the images premultiplied. According to Apple, that code should load non-premultiplied images. So, I opened a bug in the Apple but tracker regarding this.

Posted 2 years ago #

RSS feed for this topic

Reply

You must log in to post.