We've been sitting on our iOS developer license for a little while since we've been fairly occupied catering to the problematic nature of our Android apps. As soon as we can get our existing products on Android to an acceptable level we'll begin porting Bubble Zing and possibly Simple Match over to iOS.
One of the major changes in our process and code was to put as much game logic in to portable C code for Bubble Zing so that, in theory, we could just wrap the game code in to iOS objective-c wrappers and retain the java wrappers for Android. Of course, this major shift in philosophy warranted a huge time undertaking on our Android ports as we essentially had to recode everything... meaning we may as well put a lot of code in to libraries and that whole thing.
On the plus side, all this "nativeification" of game code has excellent performance indicators and should increase our product exposure to older devices that needed all the speedups (fairly badly might I add).
I suppose this process was inevitable anyways. I mean after all, Bubble Zing 1.0.3 (the latest public version as of this writing), showed that just drawing a background graphic during gameplay - JUST DRAWING THE BACKGROUND - took 60% of the processor in several test devices. That was horrendous and completely unavoidable due to the nature of java, dalvikvm, and opengl es. The only way around it was to go native code and start screwing with lower level stuff... and this has already paid off in huge ways... but it was not without its own drawbacks and tradeoffs. Specifically, things that use to work great now require extensive rework and compromises. Some things now look "worse" than in 1.0.3, but perform a ton better, yet the overall gameplay experience has been increased a lot.
Anyways, once we get Bubble Zing 1.1.0 up and in to public hands, our iOS adventure shall begin! We took a look at XCode and their iOS stuff already, and I must say it was quite welcoming from a developer's perspective and I'm going to assume from a business perspective as well once we get there!
Hopefully from here on out, all our products will see a simultaneous Android and iOS release at the same time!
Friday, April 27, 2012
Saturday, April 14, 2012
Complexities of Android development
As an indie mobile developer, I can personally attest that making things on Android - especially games - is pretty difficult.
I ran across this article about Battleheart's developer basically giving up on the Android platform as being "unsustainable" (original blog post by the developer here).
Now, this person has quite a lot of good points that works against Android devices in the business sense. Why would you want to support something that takes a vast disproportionate amount of money to create?
Android has lots of problems for developers, and one of which is that simply Android users are generally more frugal with buying things (to include newer devices). This means that as a developer, we should cater to old technology because a decent segment of the Android market are using old devices.
For example, I attempt to target Android 1.5 devices when and where possible so that, in theory, all newer devices should be able to run our games. That sounds good at first, but this isn't always the reality, and this is where Battleheart's developer has a good point. The only sure-fire way to guarantee that apps will work on a device is to physically test it on that device.
I wish it was an easy case were we can just load up our apps in an emulator, test, fix, deploy, but it definitely isn't that simple. Each device manufacturer seems to have implemented various things about Android in their own way; forcing developers to cater to each device maker!
For example, only Android 2.1 allows for multitouch. We created a virtual Dpad app that accepts user input on a circle (simulated analog dpad if you will) along with 2 virtual "buttons" - think like a Nintendo controller, but with a virtual directional pad.
After several days of developing and testing, it works 100% on our Motorola RAZR, but breaks horribly on Samsung Galaxy Tablet... and it all came down to the fact that Samsung implemented multitouch differently than Motorola (Motorola seemingly did it in accordance with the Android dev specs... Samsung apparently didn't?!). As a developer, there would be no way for me to know this unless I physically had both of these devices... and this is a major problem with deploying games/apps on Android without having hundreds/thousands of Android devices to test against. As a small indie shop, this means very unlikely to fork up all this cash for devices just to get meager amounts of downloads in Google Play market.
I can definitely feel what Mika Mobile is talking about as we've run in to these barriers as well. It isn't impossible, but it sure raises the bar for "quality" (ie, not crashing/breaking) apps to have widespread deployment on the Android platform.
I ran across this article about Battleheart's developer basically giving up on the Android platform as being "unsustainable" (original blog post by the developer here).
Now, this person has quite a lot of good points that works against Android devices in the business sense. Why would you want to support something that takes a vast disproportionate amount of money to create?
Android has lots of problems for developers, and one of which is that simply Android users are generally more frugal with buying things (to include newer devices). This means that as a developer, we should cater to old technology because a decent segment of the Android market are using old devices.
For example, I attempt to target Android 1.5 devices when and where possible so that, in theory, all newer devices should be able to run our games. That sounds good at first, but this isn't always the reality, and this is where Battleheart's developer has a good point. The only sure-fire way to guarantee that apps will work on a device is to physically test it on that device.
I wish it was an easy case were we can just load up our apps in an emulator, test, fix, deploy, but it definitely isn't that simple. Each device manufacturer seems to have implemented various things about Android in their own way; forcing developers to cater to each device maker!
For example, only Android 2.1 allows for multitouch. We created a virtual Dpad app that accepts user input on a circle (simulated analog dpad if you will) along with 2 virtual "buttons" - think like a Nintendo controller, but with a virtual directional pad.
After several days of developing and testing, it works 100% on our Motorola RAZR, but breaks horribly on Samsung Galaxy Tablet... and it all came down to the fact that Samsung implemented multitouch differently than Motorola (Motorola seemingly did it in accordance with the Android dev specs... Samsung apparently didn't?!). As a developer, there would be no way for me to know this unless I physically had both of these devices... and this is a major problem with deploying games/apps on Android without having hundreds/thousands of Android devices to test against. As a small indie shop, this means very unlikely to fork up all this cash for devices just to get meager amounts of downloads in Google Play market.
I can definitely feel what Mika Mobile is talking about as we've run in to these barriers as well. It isn't impossible, but it sure raises the bar for "quality" (ie, not crashing/breaking) apps to have widespread deployment on the Android platform.
Wednesday, April 11, 2012
Getting the "density" of your Android device
This is one of those crazy small things you'll be needing sooner or later in game development on Android devices.
How to get the "density" of your device? Or more specifically, how many physical pixels are there per inch, how big is my display?
Well, first of all, we need to get data from our device.
That will fill in everything we will need to know. From there, we can get all sorts of useful information.
How many pixels can we work with horizontally and vertically? *Warning*: These are filled in based on the current orientation of the device!!!
How about pixels per inch on the display?
If we know how many pixels are given in a dimension, and how many pixels there are per inch, we can determine the exact inches of our display (more or less).
Now, for the part that can be a little bit more mysterious. What about the actual "density"? Well, in Android API 4+, our DisplayMetric (dm) has a property conveniently called "densityDpi". This is well and good for Android 1.6 and beyond, but what about our poor Android 1.5 customers where this field didn't exist? Fear not, we can still get this information!
There is a field called "density" which is a floating number based on the scaling factor of our pixels in comparison to a 160 DPI device. In other words, if our device is "160 dpi", this scaling factor is 1 (160/160 = 1).
Common DPIs for Android devices are: 120, 160, 240, and 320. Knowing this, we can now cater to Android 1.5's missing densityDpi field by making our own.
Normally it is NOT a good idea to do a straight floating point equality comparison check (rounding errors, precision point problems, etc), but in this case, there shouldn't be any issue.
Correction: Do not do it this way. We recently received a problem report from a user with a 148 density, which is seemingly not standard. I will leave this code in here for illustration and so that the reader gets an idea of catering to different hardware displays.
The following way is how you should calculate the density.
*Update* 4.29.2012 : Added the note about a user with a 148 density.
How to get the "density" of your device? Or more specifically, how many physical pixels are there per inch, how big is my display?
Well, first of all, we need to get data from our device.
DisplayMetrics dm = new DisplayMetrics();
getWindowManager().getDefaultDisplay().getMetrics(dm);
That will fill in everything we will need to know. From there, we can get all sorts of useful information.
How many pixels can we work with horizontally and vertically? *Warning*: These are filled in based on the current orientation of the device!!!
xpixels = dm.widthPixels;
ypixels = dm.heightPixels;
How about pixels per inch on the display?
xpixelsPerInch = dm.xdpi;
ypixelsPerInch = dm.ydpi;
If we know how many pixels are given in a dimension, and how many pixels there are per inch, we can determine the exact inches of our display (more or less).
displayWidth = xpixels / xpixelsPerInch;
displayHeight = ypixels / ypixelsPerInch;
Now, for the part that can be a little bit more mysterious. What about the actual "density"? Well, in Android API 4+, our DisplayMetric (dm) has a property conveniently called "densityDpi". This is well and good for Android 1.6 and beyond, but what about our poor Android 1.5 customers where this field didn't exist? Fear not, we can still get this information!
There is a field called "density" which is a floating number based on the scaling factor of our pixels in comparison to a 160 DPI device. In other words, if our device is "160 dpi", this scaling factor is 1 (160/160 = 1).
Common DPIs for Android devices are: 120, 160, 240, and 320. Knowing this, we can now cater to Android 1.5's missing densityDpi field by making our own.
if (dm.density == 0.75)
densityDpi = 120;
if (dm.density == 1.0)
densityDpi = 160;
if (dm.density == 1.5)
densityDpi = 240;
if (dm.density == 2)
densityDpi = 320;
Normally it is NOT a good idea to do a straight floating point equality comparison check (rounding errors, precision point problems, etc), but in this case, there shouldn't be any issue.
Correction: Do not do it this way. We recently received a problem report from a user with a 148 density, which is seemingly not standard. I will leave this code in here for illustration and so that the reader gets an idea of catering to different hardware displays.
The following way is how you should calculate the density.
densityDpi = (int) (dm.density * 160.0f);
*Update* 4.29.2012 : Added the note about a user with a 148 density.
Monday, April 2, 2012
So many Android devices to cater to...
In follow up to my earlier post about being envious of software houses of yesteryear, I have to admit that in a sense, they had it easier.
When they made their games and products, they had a single target system with known specs and limitations. Code that relied on *timing* of events and things like this can be planned for meticulously. Developers also knew 100% ahead of time what features a system did or did not have. Yet, they also dealt with 8-bit / 16-bit processors, no FPU, no advanced graphics pipeline, and all sorts of "limitations", and they still cranked out great products after great products all without needing all these patches and what not.
Well, fast forward a bunch of years, and here we're in the age of the smartphone. In some regards, we have it easy - a powerful CPU (relatively speaking), an amazing graphics processor, and sometimes a FPU. Wait, sometimes?! Yes, that's right, if you're a developer for the Android platform, you can't be guaranteed to have a floating point unit. You also have no guarantee of having any given amount of RAM, disk space, or really much of anything. Sure, you can *use* float data types, but on devices with no dedicated float processor, it is software emulated!!! Talk about a massive performance hit if you want to do updates and renders at any sane framerate!
Our earlier game, Bubble Zing, ran amazingly awesome on newer Android phones, but on older phones, it ran terribly. With the 1.0.3 release of the game last month, we thought all major compatibility issues were resolved; well, that was only partially correct. The major bugs seemingly were ironed out, and the core game "worked", but with no guarantee of how well it would work - sometimes outright unplayable on some devices!
So, we delve further in to the "why is this running like terrible on x,y,z,q,f devices, but great on others". Well, this is where the things Android does is both great and bad at the same time. We coded 1.0.3 in 100% Java as this was seemingly the correct way to make any app on Android devices. In theory, it sounded good since Java apps are suppose to work 100% on devices with a compatible JVM (ie, all Android devices) - clearly something we wanted, and it is fairly easy to crank a Java program much faster than an equivalent C/C++ program, so it sounded like a great idea at the time.
Our test system was a Droid RAZR with Android 2.3.6. Our initial testing was running well, FPS was over 40 and everything seemed great. The emulator running an AVD 1.5 seemed sluggish but that could be written off as being an emulator and running not on the most powerful host computer.
Things got really confusing and exciting when we tested 1.0.1 and 1.0.2 on a Samsung Galaxy tablet -- that's when all hell broke loose. Everything that should have been working in theory wasn't. On top of that, the frame rate was a dismal 10 to 15! What the flying expletive! How can a program go from 40+ FPS to 10! It wasn't an issue with the version of the Android OS, it must have been something device specific.
I ran a traceview to track what was eating all the processing time and it came down to rendering a full screen background PNG file. On the phone, this wasn't such a big deal, but on the tablet, it was eating almost 60% of all CPU time!
We're sitting here with a complete Java game that worked ok on an emulator and great on our target test phone, yet are dumbfounded with why the same exact Java code on the tablet ran so terribly.
A lot of trial and error later we figured that to get things working faster and better, the only way was to ditch Java and start programming on a closer-to-the-hardware level as we could to squeeze out a lot more performance and manage memory more effectively. Java made it very frustrating with garbage collecting at some fairly inconvenient times and this compounded other issues making the game run very badly at certain instances. It was time to bring on the NDK.
Frustration is what is in store for Android developers. It can be done, but good apps will need a very persistent set of people to make these products and make them well.
Oh, and I'm still in awe of game programmers from decades ago on consoles, but at the same time, I'm jealous of their "single architecture" deployment platform as that probably simplified a lot of these problems thousands fold. I suppose this is another reason Apple is winning the smartphone wars with developers as it is "easier" to make apps on iOS with very heavily controlled hardware features by Apple.
When they made their games and products, they had a single target system with known specs and limitations. Code that relied on *timing* of events and things like this can be planned for meticulously. Developers also knew 100% ahead of time what features a system did or did not have. Yet, they also dealt with 8-bit / 16-bit processors, no FPU, no advanced graphics pipeline, and all sorts of "limitations", and they still cranked out great products after great products all without needing all these patches and what not.
Well, fast forward a bunch of years, and here we're in the age of the smartphone. In some regards, we have it easy - a powerful CPU (relatively speaking), an amazing graphics processor, and sometimes a FPU. Wait, sometimes?! Yes, that's right, if you're a developer for the Android platform, you can't be guaranteed to have a floating point unit. You also have no guarantee of having any given amount of RAM, disk space, or really much of anything. Sure, you can *use* float data types, but on devices with no dedicated float processor, it is software emulated!!! Talk about a massive performance hit if you want to do updates and renders at any sane framerate!
Our earlier game, Bubble Zing, ran amazingly awesome on newer Android phones, but on older phones, it ran terribly. With the 1.0.3 release of the game last month, we thought all major compatibility issues were resolved; well, that was only partially correct. The major bugs seemingly were ironed out, and the core game "worked", but with no guarantee of how well it would work - sometimes outright unplayable on some devices!
So, we delve further in to the "why is this running like terrible on x,y,z,q,f devices, but great on others". Well, this is where the things Android does is both great and bad at the same time. We coded 1.0.3 in 100% Java as this was seemingly the correct way to make any app on Android devices. In theory, it sounded good since Java apps are suppose to work 100% on devices with a compatible JVM (ie, all Android devices) - clearly something we wanted, and it is fairly easy to crank a Java program much faster than an equivalent C/C++ program, so it sounded like a great idea at the time.
Our test system was a Droid RAZR with Android 2.3.6. Our initial testing was running well, FPS was over 40 and everything seemed great. The emulator running an AVD 1.5 seemed sluggish but that could be written off as being an emulator and running not on the most powerful host computer.
Things got really confusing and exciting when we tested 1.0.1 and 1.0.2 on a Samsung Galaxy tablet -- that's when all hell broke loose. Everything that should have been working in theory wasn't. On top of that, the frame rate was a dismal 10 to 15! What the flying expletive! How can a program go from 40+ FPS to 10! It wasn't an issue with the version of the Android OS, it must have been something device specific.
I ran a traceview to track what was eating all the processing time and it came down to rendering a full screen background PNG file. On the phone, this wasn't such a big deal, but on the tablet, it was eating almost 60% of all CPU time!
We're sitting here with a complete Java game that worked ok on an emulator and great on our target test phone, yet are dumbfounded with why the same exact Java code on the tablet ran so terribly.
A lot of trial and error later we figured that to get things working faster and better, the only way was to ditch Java and start programming on a closer-to-the-hardware level as we could to squeeze out a lot more performance and manage memory more effectively. Java made it very frustrating with garbage collecting at some fairly inconvenient times and this compounded other issues making the game run very badly at certain instances. It was time to bring on the NDK.
Frustration is what is in store for Android developers. It can be done, but good apps will need a very persistent set of people to make these products and make them well.
Oh, and I'm still in awe of game programmers from decades ago on consoles, but at the same time, I'm jealous of their "single architecture" deployment platform as that probably simplified a lot of these problems thousands fold. I suppose this is another reason Apple is winning the smartphone wars with developers as it is "easier" to make apps on iOS with very heavily controlled hardware features by Apple.
Thursday, March 29, 2012
Android NDK loading resources and PNGs
After following the nice code sample from http://androgeek.info/?p=275 about how to load a PNG file by accessing the Android APK file directly using only the NDK, everything seemed awesome.
It uses libzip and libpng to manually load the APK, then load a png file right from the APK itself without using any Java or the Android SDK.
Why would you want to do this? Well, aside from liking complicated stuff, it is useful for us because now we can put all of our "portable" resource loading code in c/c++ files so it is more easily moved to iOS, windows, whatever, without being bound to using only the Android SDK, thus having to recode loading logic over and over again for each target deployment architecture.
Everything worked well, until something didn't go quite right. Some of my textures were corrupted and worse, some actually crashed my app!
It took 4 days to track down what exactly was happening.
Common problems usually indicate I was doing something wrong with opengl and the textures themselves...
* All my textures were perfect powers of two (64x64, 128x128, etc)
* All my textures were 32-bit RGBA files
* I fixed a warning and error from the given sample code to correctly read the libzip'd apk file
So, it was time to dig deeper with what was going on.
First, I noticed that my glGenTextures calls were giving me very large opengl texture ids so I thought this was a huge problem as it was not continuous numbers, and in C, any time you have generating wild numbers, it usually means a bad pointer somewhere. I spent about a day analyzing and referencing code and started to realize that the generated numbers were always the same (this usually rules out a memory corruption issue if the results are *always* the same).
I noticed that the generated opengl texture ids always followed this pattern on my Droid RAZR:
100271
315638026
534244737
1505553200
-1563003837
...
etc.
Yet, on my Android emulator, the texture id's were:
1
2
3
4
5
...
etc
I had found this thread on google groups that I thought was my problem since it sounded really similar, but it didn't actually help.
Knowing that glGenTextures expects an *unsigned* integer as the data type, I realized that the negative number was an overflowed signed integer and actually nothing to worry about. In my debugging print code I needed to change "%d" to "%u" to see the actual unsigned value.
Since these numbers repeated on every subsequent run of the program, I decided to test the idea that maybe my code had nothing at all to do with the "corrupt" large texture ids. Indeed, I made a program that literally only called glGenTextures about 100 times right at startup time and did nothing else. The result? The exact same large numbers showed up in my device, and was "normal" on the emulator, as expected. This left me with the conclusion that whatever logic is in my Motorola device in terms of picking a unique texture ID that this was normal - next.
Yet, this didn't answer my question as to why I was seeing an app crash occur and seemingly only when loading a certain texture or two. My first order of business was to rule out if my other code was at fault, or was it somehow related to ONLY those sets of textures (somehow). Strangely, dozens of tests indicated that it was only the problem of 3 specific textures being loaded as I added lots of additional textures, even loaded some textures multiple times, then finally ONLY loading one of those 3 problematic textures. The app would only crash at loading any of the 3.
Frustrated, I ended up putting a ton of extra debugging code all over my apps trying to track down what in the world was wrong. All indicators were pointing to the idea that a problem was with either libzip or libpng since libc.so itself was crashing in a call to a libpng function -- great, time to track down potential bugs in those libraries.
Now, let me put a disclaimer up here, it is almost never a good idea to assume a mature code library has a bug in it; especially one this severe, yet here I am with a consistent crash always calling a specific line of code that libpng was using leaving me to believe this could be the case.
First, I downloaded the most recent version of each library and replaced the old versions that were provided in the linked post above. I was now using libzip 0.10.1 and libpng 1.5.9. This had promise since each of those had a bit of fixes/enhancements along the way. I was hopeful that this would magically fix my problems.
Well, it did - sort of.
Whatever wizardry was happening internally in those libraries helped indicate that libpng was crashing specifically on this line:
png_read_info(png_ptr, info_ptr);
Great, all this work only to help reinforce the idea that libpng was broken for my special case. Yet, I still couldn't rule out that libzip somehow corrupted my image data while decompressing my APK so libpng might not be at fault trying to load corrupted data. Worse, all error checking came up empty for all libzip and libpng calls! I even checked my opengl calls for errors using glError -- NOTHING!
Well, the next step was to dive in to those library's source code and start tracking stuff down, the last resort I guess was to find the problem in one of these libraries and fix it myself and give the respective team a patch!
One of my testing tangents included making a stand alone C project to test very specific sets of code to completely rule out any Android Java JNI mysticism causing problems. Everything was pointing to only those textures being a problem in the libpng loading code. So I ran the libpng test program on those 3 test pngs and the only thing that sounded remotely threatening was:
Files aaaa.png and pngout.png are different
Was aaaa.png written with the same maximum IDAT chunk size (8192 bytes), filtering heuristic (libpng default), compression level (zlib default), and zlib version (1.2.5)?
It certainly sounded menacing and I was convinced that somehow my textures were corrupted. I ended up regenerating those textures using the latest version of Gimp (2.6.11 at the time) -- plugged those in to my app in excitement; and they still crashed.
Strangely, the libpng test program generated pngout.png which was seemingly the same exact copy of my original texture, but a different filesize; something was different, and different was good at this point. For giggles, I put that pngout.png file in to my Android's APK res/raw folder in place of my original texture and much to my surprise the texture loaded!!! Ah-hah, the answer finally!
Unfortunately, things weren't over yet, the texture loaded completely corrupted. Everything was misaligned, the colors were wrong and the texture was completely garbled with random colors all over it. Basically, I've gone nowhere.
3 days in and I'm still struggling with what is wrong here. I'm still baffled, why is it that my 10 other textures work fine, yet these 3 don't want to work at all?! I decided to write my own texture loading function from scratch to ensure I understood exactly every step of logic that was taking place. The same result happened. I did all sorts of strange additional tests, hex dumps, you name it. I was at the end of my rope on this one, I was getting angry.
Finally, I saw an interesting article somewhere that said I needed to rename my png files to mp3 files. I thought to myself, that's the dumbest thing I ever heard - why would I ever want to do that?! Well, if you're programming for Android, you might be crazy enough to hear this out.
Apparently, during the APK packing process *SOME* of your png textures are automagically compressed and others are not. Renaming your png files tells the apk packing process to not mangle/compress that file, and my happy 3 textures were some of the lucky selected ones for this process; which completely broke the texture loading code. In a move of desperation, I did rename my files to mp3 to verify this was the case, and they worked perfectly. I was enraged and ecstatic at the same time.
On the bright side I got a better understanding of both libpng and libzip (we got a little familiar with each other, if you know what I mean).
Alright, so the magic bullet answer is that I need to rename all my PNG files to MP3 (test.png.mp3 if you will) or simply dump them all in to a game data file of sorts (probably the better idea).
So, I make a quick resource packing program (using zlib) and I find out that during the APK packing process again, my GZ files are also modified!!!
I quickly found this link explaining it a little bit.
I wonder what other goodies are in store for me to discover by accident!?
Hope this article is found by anyone else stuck in my situation and save you all a couple days of madness.
It uses libzip and libpng to manually load the APK, then load a png file right from the APK itself without using any Java or the Android SDK.
Why would you want to do this? Well, aside from liking complicated stuff, it is useful for us because now we can put all of our "portable" resource loading code in c/c++ files so it is more easily moved to iOS, windows, whatever, without being bound to using only the Android SDK, thus having to recode loading logic over and over again for each target deployment architecture.
Everything worked well, until something didn't go quite right. Some of my textures were corrupted and worse, some actually crashed my app!
It took 4 days to track down what exactly was happening.
Common problems usually indicate I was doing something wrong with opengl and the textures themselves...
* All my textures were perfect powers of two (64x64, 128x128, etc)
* All my textures were 32-bit RGBA files
* I fixed a warning and error from the given sample code to correctly read the libzip'd apk file
So, it was time to dig deeper with what was going on.
First, I noticed that my glGenTextures calls were giving me very large opengl texture ids so I thought this was a huge problem as it was not continuous numbers, and in C, any time you have generating wild numbers, it usually means a bad pointer somewhere. I spent about a day analyzing and referencing code and started to realize that the generated numbers were always the same (this usually rules out a memory corruption issue if the results are *always* the same).
I noticed that the generated opengl texture ids always followed this pattern on my Droid RAZR:
100271
315638026
534244737
1505553200
-1563003837
...
etc.
Yet, on my Android emulator, the texture id's were:
1
2
3
4
5
...
etc
I had found this thread on google groups that I thought was my problem since it sounded really similar, but it didn't actually help.
Knowing that glGenTextures expects an *unsigned* integer as the data type, I realized that the negative number was an overflowed signed integer and actually nothing to worry about. In my debugging print code I needed to change "%d" to "%u" to see the actual unsigned value.
Since these numbers repeated on every subsequent run of the program, I decided to test the idea that maybe my code had nothing at all to do with the "corrupt" large texture ids. Indeed, I made a program that literally only called glGenTextures about 100 times right at startup time and did nothing else. The result? The exact same large numbers showed up in my device, and was "normal" on the emulator, as expected. This left me with the conclusion that whatever logic is in my Motorola device in terms of picking a unique texture ID that this was normal - next.
Yet, this didn't answer my question as to why I was seeing an app crash occur and seemingly only when loading a certain texture or two. My first order of business was to rule out if my other code was at fault, or was it somehow related to ONLY those sets of textures (somehow). Strangely, dozens of tests indicated that it was only the problem of 3 specific textures being loaded as I added lots of additional textures, even loaded some textures multiple times, then finally ONLY loading one of those 3 problematic textures. The app would only crash at loading any of the 3.
Frustrated, I ended up putting a ton of extra debugging code all over my apps trying to track down what in the world was wrong. All indicators were pointing to the idea that a problem was with either libzip or libpng since libc.so itself was crashing in a call to a libpng function -- great, time to track down potential bugs in those libraries.
Now, let me put a disclaimer up here, it is almost never a good idea to assume a mature code library has a bug in it; especially one this severe, yet here I am with a consistent crash always calling a specific line of code that libpng was using leaving me to believe this could be the case.
First, I downloaded the most recent version of each library and replaced the old versions that were provided in the linked post above. I was now using libzip 0.10.1 and libpng 1.5.9. This had promise since each of those had a bit of fixes/enhancements along the way. I was hopeful that this would magically fix my problems.
Well, it did - sort of.
Whatever wizardry was happening internally in those libraries helped indicate that libpng was crashing specifically on this line:
png_read_info(png_ptr, info_ptr);
Great, all this work only to help reinforce the idea that libpng was broken for my special case. Yet, I still couldn't rule out that libzip somehow corrupted my image data while decompressing my APK so libpng might not be at fault trying to load corrupted data. Worse, all error checking came up empty for all libzip and libpng calls! I even checked my opengl calls for errors using glError -- NOTHING!
Well, the next step was to dive in to those library's source code and start tracking stuff down, the last resort I guess was to find the problem in one of these libraries and fix it myself and give the respective team a patch!
One of my testing tangents included making a stand alone C project to test very specific sets of code to completely rule out any Android Java JNI mysticism causing problems. Everything was pointing to only those textures being a problem in the libpng loading code. So I ran the libpng test program on those 3 test pngs and the only thing that sounded remotely threatening was:
Files aaaa.png and pngout.png are different
Was aaaa.png written with the same maximum IDAT chunk size (8192 bytes), filtering heuristic (libpng default), compression level (zlib default), and zlib version (1.2.5)?
It certainly sounded menacing and I was convinced that somehow my textures were corrupted. I ended up regenerating those textures using the latest version of Gimp (2.6.11 at the time) -- plugged those in to my app in excitement; and they still crashed.
Strangely, the libpng test program generated pngout.png which was seemingly the same exact copy of my original texture, but a different filesize; something was different, and different was good at this point. For giggles, I put that pngout.png file in to my Android's APK res/raw folder in place of my original texture and much to my surprise the texture loaded!!! Ah-hah, the answer finally!
Unfortunately, things weren't over yet, the texture loaded completely corrupted. Everything was misaligned, the colors were wrong and the texture was completely garbled with random colors all over it. Basically, I've gone nowhere.
3 days in and I'm still struggling with what is wrong here. I'm still baffled, why is it that my 10 other textures work fine, yet these 3 don't want to work at all?! I decided to write my own texture loading function from scratch to ensure I understood exactly every step of logic that was taking place. The same result happened. I did all sorts of strange additional tests, hex dumps, you name it. I was at the end of my rope on this one, I was getting angry.
Finally, I saw an interesting article somewhere that said I needed to rename my png files to mp3 files. I thought to myself, that's the dumbest thing I ever heard - why would I ever want to do that?! Well, if you're programming for Android, you might be crazy enough to hear this out.
Apparently, during the APK packing process *SOME* of your png textures are automagically compressed and others are not. Renaming your png files tells the apk packing process to not mangle/compress that file, and my happy 3 textures were some of the lucky selected ones for this process; which completely broke the texture loading code. In a move of desperation, I did rename my files to mp3 to verify this was the case, and they worked perfectly. I was enraged and ecstatic at the same time.
On the bright side I got a better understanding of both libpng and libzip (we got a little familiar with each other, if you know what I mean).
Alright, so the magic bullet answer is that I need to rename all my PNG files to MP3 (test.png.mp3 if you will) or simply dump them all in to a game data file of sorts (probably the better idea).
So, I make a quick resource packing program (using zlib) and I find out that during the APK packing process again, my GZ files are also modified!!!
I quickly found this link explaining it a little bit.
I wonder what other goodies are in store for me to discover by accident!?
Hope this article is found by anyone else stuck in my situation and save you all a couple days of madness.
Subscribe to:
Posts (Atom)