Sapphire Benchmarks

Discussion in 'cvpcs' started by Homindroid, Sep 19, 2010.

  1. Homindroid
    Offline

    Homindroid New Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Virginia, near the Beach
    I have posted a few Sapphire 1.1.1 benchmarks on my blog. Check it out and if you have any suggestions please pass them my way!!

    http://homindroid.wordpress.com
  2. sniffle
    Offline

    sniffle Premium Member Premium Member Developer

    Joined:
    Apr 6, 2010
    Messages:
    291
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Birmingham AL
    ok, since you are going through all of the trouble to test out all of these kernels, you might want to account for all of the variables that can change the results, to make your benchmark tests truly accurate. because on your device without those variables being defined the benchmark scores are nothign more than numbers that no one else can duplicate.

    you need things like apps installed, apps opened in background, setcpu settings, ~temp+-X degrees, specific kernel downloaded and installed, voltages used in those kernels, speed settigns for each slot in said kernels, etc.

    I like what you are doing, because a true comparison needs to be made between kernels. but a benchmark alone without info for other people to be able to duplicate your results is useless. as it only says hey guys here is what my phone can do. take the scientific approach, define every variable you can possibly think of, make those variables as consistent as possible between each test, and then post your results. because then any joker that comes along is able to duplicate your set variables and compare his phone's performance vs your phone's performance, as each chip is unique it will also give people a good judge of how "perfectly" made their processor is.

    a good person to reference should you choose to persue this fully as i have detailed is Skull_One from the overclocking 101 thread, he has both CS and EE degrees. he can name more variables that need to be defined probably.
  3. pdroid
    Offline

    pdroid New Member

    Joined:
    Mar 4, 2010
    Messages:
    2,252
    Likes Received:
    7
    Trophy Points:
    0
    Location:
    Philly
    Stock kernel in Sapphire 1.1.1 is 250-800 MHz, standard voltage. You have this mislabeled in your post as 1 GHz. I was confused at first as to why the stock kernel would take so much longer to calculate pi (the other kernels all give roughly the same result, as one would expect).
  4. sniffle
    Offline

    sniffle Premium Member Premium Member Developer

    Joined:
    Apr 6, 2010
    Messages:
    291
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Birmingham AL
    pdroid you should put in your signature that you are an EE, so that on things related to kernels people know that you know what you are talking about... :p
  5. Homindroid
    Offline

    Homindroid New Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Virginia, near the Beach
    Sniffle,

    Thankyou for the excellent reply! I am always game to improve the process and make the benchmarks as analytical as possible!

    As far as programs go, I only install what I need to perform screen captures (drocap) and the benchmarks themselves. That said, I WILL now document what apps and processes are running while performing the benchmarks. I also have chosen 1 ghz as the standard and only kernels listed as "low voltage" are maked as such. (IE- ChevyNo1 LV 1ghz). All kernels are curerently downloaded from ROM Manager with the exception of P3's kernal, which is downloaded from his site. I will looking to provide links to the specific kernels I use.

    I will try and get in touch with Skull_One as see if he will give me some guidance on how to "tighten" things up in regards to the benchmarks.

    Again, thanks for the critique and info!

    Dustin

    edit to add: I will be updating my procedures blog post to fully reflect how I perform these benchmarks. I think it will make the process easier to tweak if all is known.
  6. Homindroid
    Offline

    Homindroid New Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Virginia, near the Beach
    PDroid, thanks! That slipped by me! Post updated.

    Dustin
  7. sniffle
    Offline

    sniffle Premium Member Premium Member Developer

    Joined:
    Apr 6, 2010
    Messages:
    291
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Birmingham AL
    no problem by doing this you will actually give people a true baseline to compare against, instead of numbers beign thrown around... it's needed to be done for a while now.
  8. pdroid
    Offline

    pdroid New Member

    Joined:
    Mar 4, 2010
    Messages:
    2,252
    Likes Received:
    7
    Trophy Points:
    0
    Location:
    Philly
    Cool - but might I recommend also including cvpcs' 1 GHz kernel in your comparisons.

    Cheers. :)
  9. Homindroid
    Offline

    Homindroid New Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Virginia, near the Beach
    PDroid, I will add that kernel later today (Hopefully). Off to work
  10. Skull One
    Offline

    Skull One New Member

    Joined:
    Mar 11, 2010
    Messages:
    760
    Likes Received:
    6
    Trophy Points:
    0

    First let me commend you for trying to test a very complicated situation. But alas, the reason I have never done this test is due to the amount of time it takes to make the test "scientifically sound" and "reproduce-able".

    Your Linpack results should have less than a .5% difference between kernels of the same speed under the same condition of testing. If we remove Sapphires 800Mhz kernel from your result, you have a 2% difference between your highest and lowest value. Since linpack is purely a CPU derived benchmark, the same CPU tested with a kernel that has the same speed should give the same exact set of results.

    Here is what is needed, in my opinion, to get a valid set of benchmarks:

    1) Install the ROM clean and DO NOT activate it.
    2) Put phone in airplane mode.
    3) Delete every application from system/app and data/app except what is needed to boot the phone and get the launcher running.
    4) Install all testing apps from a SD card and then remove that SD card.
    5) Disable all apps from auto-starting under any condition.
    6) Use the performance governor.
    7) Each test should be running a minimum of 10 times to make sure your variance in testing is less than .5%. If it is greater, you left something running from either step 3 or 5.

    Now if you want to test battery runs, you have to go even further.
    1) Fully charge the battery with a wall charger.
    2) Delete /data/system/batterystats.bin.
    3) Start a battery tracker with 1 minute logging.
    4) Set the screen to 100% power and no timeout.
    5) Start the test.
    6) Unplug the phone from the charger.
    7) Wait for the battery to read a minimum of 70%.
    8) Stop the test.
    9) Pull the log data for the battery and use the data from 90% down to 80%.

    One day I will find the time to do this kind of detailed testing. But so far, the state of ROM development has precluded me from doing. Simply put, ROM development is still too much of a moving target at present.
  11. Homindroid
    Offline

    Homindroid New Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Virginia, near the Beach
    Skull_One,

    I will adopt these procedures for all future benchmarks. Noob Question...Does anyone know what apps must be running to boot properly? (Step 3) and how to disable apps from autostarting? Everything else seems straight forward.

    Dustin
  12. Skull One
    Offline

    Skull One New Member

    Joined:
    Mar 11, 2010
    Messages:
    760
    Likes Received:
    6
    Trophy Points:
    0
    The last time I did a wipe of all apps was back in the 2.1 days. So I don't have a current list. If I get a chance in the next day or so, I will try to compile a list for the Sapphire ROM.

    I use autostarts from the market to keep apps from being able to auto launch based on system events.
  13. Skull One
    Offline

    Skull One New Member

    Joined:
    Mar 11, 2010
    Messages:
    760
    Likes Received:
    6
    Trophy Points:
    0
    I have started making a clean room ROM. Got a baseline install up and running, tonight or tomorrow during my lunch break, I will start removing the remaining apps and lock down the apps that can auto start.

    Homindroid,
    If you use Clockwork Mod for backup/restores, when I finish getting it to "clean room" status, do you want a copy of it?
  14. Homindroid
    Offline

    Homindroid New Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Virginia, near the Beach
    Skull_One,

    Definately! I have already backup'd my the apps I need for benchmarks to my sd card and bought the "autostarts" app. One though, If I do not activate or sign in, how do I restore those apps from the sd card? Do I push a file manager through ADB? Her is the apps I plan on using;

    • Benchmark PI
    • CPU Benchmark
    • Linpack
    • Quadrant
    • autostarts
    • astro/root explorer (maybe?)
    • Quadrant
    Ofcourse, if you think some of these are redundant or have another suggestion, I'm game.

    I really can't wait to get started with the new "Clean Room" Sapphire! I really appreciate all the help!

    I'm going back to work, teaching till 4. I'll check this thread around 3pm.

    Dustin
  15. Skull One
    Offline

    Skull One New Member

    Joined:
    Mar 11, 2010
    Messages:
    760
    Likes Received:
    6
    Trophy Points:
    0
    I push Root Explorer right off the bat. Then I install autostarts, SetCPU, Battery Graph, drocap2, OSMonitor and aLogcat. With that baseline installed, I can then kill off the remaining apps that aren't needed for booting, kill the auto launch stuff and then with OSmonitor and aLogcat confirm that the phone is nearly brain dead ;)
  16. Skull One
    Offline

    Skull One New Member

    Joined:
    Mar 11, 2010
    Messages:
    760
    Likes Received:
    6
    Trophy Points:
    0
    I have a baseline Sapphire 1.1.1 Clean Room ROM set up. But I am having issues with installing OSMonitor on it to confirm it is truly clean. Soon as I figure out why it is failing to install, I can turn it over.
  17. Homindroid
    Offline

    Homindroid New Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Virginia, near the Beach
    Sounds good... I was fooling around and deleting apps from
    /system/apps...ya... learned not to delete phone.apk...lol

    Posted via Tapatalk
  18. Skull One
    Offline

    Skull One New Member

    Joined:
    Mar 11, 2010
    Messages:
    760
    Likes Received:
    6
    Trophy Points:
    0

    And there is a reason for that. If the Launcher that is marked as "Home" has crashed and can't be restarted properly, it defaults to phone.apk
  19. Homindroid
    Offline

    Homindroid New Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Virginia, near the Beach
    I have added some new kernel benchmarks on the blog. Check it out and let me know if you see issues...

    Homindroid's Benchmarks

    Dustin
  20. Skull One
    Offline

    Skull One New Member

    Joined:
    Mar 11, 2010
    Messages:
    760
    Likes Received:
    6
    Trophy Points:
    0
    First, I want to thank you for the funniest thing that has happened to me all week! I think I laughed for a good solid minute after reading my call sign as Kill_One. In fact, I may have to reserve that call sign here just because of how funny I found that! :)

    I am glad to see the results of linpack tightened up to where I knew they could be based on my own testing. That gives me a great deal of faith that the rest of your tests/results are now spot on. Well done!

    And my apologizes for not realizing that Quadrant needed WiFi to get a result. I am going to work on the Clean Room ROM to see if I can get WiFi back online without sacrificing testing consistency.


    Who makes the "CPU Benchmark" app, so I can try to include it in the Clean Room ROM?
Search tags for this page

benmark pi apk