Sapphire Benchmarks

Discussion in 'cvpcs' started by Homindroid, Sep 19, 2010.

  1. Homindroid

    Homindroid Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Virginia, near the Beach
    Ratings:
    +0
    I have posted a few Sapphire 1.1.1 benchmarks on my blog. Check it out and if you have any suggestions please pass them my way!!

    http://homindroid.wordpress.com
     
  2. sniffle

    sniffle Premium Member Premium Member Developer

    Joined:
    Apr 6, 2010
    Messages:
    291
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Birmingham AL
    Ratings:
    +0
    ok, since you are going through all of the trouble to test out all of these kernels, you might want to account for all of the variables that can change the results, to make your benchmark tests truly accurate. because on your device without those variables being defined the benchmark scores are nothign more than numbers that no one else can duplicate.

    you need things like apps installed, apps opened in background, setcpu settings, ~temp+-X degrees, specific kernel downloaded and installed, voltages used in those kernels, speed settigns for each slot in said kernels, etc.

    I like what you are doing, because a true comparison needs to be made between kernels. but a benchmark alone without info for other people to be able to duplicate your results is useless. as it only says hey guys here is what my phone can do. take the scientific approach, define every variable you can possibly think of, make those variables as consistent as possible between each test, and then post your results. because then any joker that comes along is able to duplicate your set variables and compare his phone's performance vs your phone's performance, as each chip is unique it will also give people a good judge of how "perfectly" made their processor is.

    a good person to reference should you choose to persue this fully as i have detailed is Skull_One from the overclocking 101 thread, he has both CS and EE degrees. he can name more variables that need to be defined probably.
     
  3. pdroid

    pdroid Silver Member

    Joined:
    Mar 4, 2010
    Messages:
    2,252
    Likes Received:
    7
    Trophy Points:
    103
    Location:
    Philly
    Ratings:
    +7
    Stock kernel in Sapphire 1.1.1 is 250-800 MHz, standard voltage. You have this mislabeled in your post as 1 GHz. I was confused at first as to why the stock kernel would take so much longer to calculate pi (the other kernels all give roughly the same result, as one would expect).
     
  4. sniffle

    sniffle Premium Member Premium Member Developer

    Joined:
    Apr 6, 2010
    Messages:
    291
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Birmingham AL
    Ratings:
    +0
    pdroid you should put in your signature that you are an EE, so that on things related to kernels people know that you know what you are talking about... :p
     
  5. Homindroid

    Homindroid Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Virginia, near the Beach
    Ratings:
    +0
    Sniffle,

    Thankyou for the excellent reply! I am always game to improve the process and make the benchmarks as analytical as possible!

    As far as programs go, I only install what I need to perform screen captures (drocap) and the benchmarks themselves. That said, I WILL now document what apps and processes are running while performing the benchmarks. I also have chosen 1 ghz as the standard and only kernels listed as "low voltage" are maked as such. (IE- ChevyNo1 LV 1ghz). All kernels are curerently downloaded from ROM Manager with the exception of P3's kernal, which is downloaded from his site. I will looking to provide links to the specific kernels I use.

    I will try and get in touch with Skull_One as see if he will give me some guidance on how to "tighten" things up in regards to the benchmarks.

    Again, thanks for the critique and info!

    Dustin

    edit to add: I will be updating my procedures blog post to fully reflect how I perform these benchmarks. I think it will make the process easier to tweak if all is known.
     
  6. Homindroid

    Homindroid Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Virginia, near the Beach
    Ratings:
    +0
    PDroid, thanks! That slipped by me! Post updated.

    Dustin
     
  7. sniffle

    sniffle Premium Member Premium Member Developer

    Joined:
    Apr 6, 2010
    Messages:
    291
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Birmingham AL
    Ratings:
    +0
    no problem by doing this you will actually give people a true baseline to compare against, instead of numbers beign thrown around... it's needed to be done for a while now.
     
  8. pdroid

    pdroid Silver Member

    Joined:
    Mar 4, 2010
    Messages:
    2,252
    Likes Received:
    7
    Trophy Points:
    103
    Location:
    Philly
    Ratings:
    +7
    Cool - but might I recommend also including cvpcs' 1 GHz kernel in your comparisons.

    Cheers. :)
     
  9. Homindroid

    Homindroid Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Virginia, near the Beach
    Ratings:
    +0
    PDroid, I will add that kernel later today (Hopefully). Off to work
     
  10. Skull One

    Skull One Member

    Joined:
    Mar 11, 2010
    Messages:
    759
    Likes Received:
    6
    Trophy Points:
    18
    Ratings:
    +6

    First let me commend you for trying to test a very complicated situation. But alas, the reason I have never done this test is due to the amount of time it takes to make the test "scientifically sound" and "reproduce-able".

    Your Linpack results should have less than a .5% difference between kernels of the same speed under the same condition of testing. If we remove Sapphires 800Mhz kernel from your result, you have a 2% difference between your highest and lowest value. Since linpack is purely a CPU derived benchmark, the same CPU tested with a kernel that has the same speed should give the same exact set of results.

    Here is what is needed, in my opinion, to get a valid set of benchmarks:

    1) Install the ROM clean and DO NOT activate it.
    2) Put phone in airplane mode.
    3) Delete every application from system/app and data/app except what is needed to boot the phone and get the launcher running.
    4) Install all testing apps from a SD card and then remove that SD card.
    5) Disable all apps from auto-starting under any condition.
    6) Use the performance governor.
    7) Each test should be running a minimum of 10 times to make sure your variance in testing is less than .5%. If it is greater, you left something running from either step 3 or 5.

    Now if you want to test battery runs, you have to go even further.
    1) Fully charge the battery with a wall charger.
    2) Delete /data/system/batterystats.bin.
    3) Start a battery tracker with 1 minute logging.
    4) Set the screen to 100% power and no timeout.
    5) Start the test.
    6) Unplug the phone from the charger.
    7) Wait for the battery to read a minimum of 70%.
    8) Stop the test.
    9) Pull the log data for the battery and use the data from 90% down to 80%.

    One day I will find the time to do this kind of detailed testing. But so far, the state of ROM development has precluded me from doing. Simply put, ROM development is still too much of a moving target at present.
     
  11. Homindroid

    Homindroid Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Virginia, near the Beach
    Ratings:
    +0
    Skull_One,

    I will adopt these procedures for all future benchmarks. Noob Question...Does anyone know what apps must be running to boot properly? (Step 3) and how to disable apps from autostarting? Everything else seems straight forward.

    Dustin
     
  12. Skull One

    Skull One Member

    Joined:
    Mar 11, 2010
    Messages:
    759
    Likes Received:
    6
    Trophy Points:
    18
    Ratings:
    +6
    The last time I did a wipe of all apps was back in the 2.1 days. So I don't have a current list. If I get a chance in the next day or so, I will try to compile a list for the Sapphire ROM.

    I use autostarts from the market to keep apps from being able to auto launch based on system events.
     
  13. Skull One

    Skull One Member

    Joined:
    Mar 11, 2010
    Messages:
    759
    Likes Received:
    6
    Trophy Points:
    18
    Ratings:
    +6
    I have started making a clean room ROM. Got a baseline install up and running, tonight or tomorrow during my lunch break, I will start removing the remaining apps and lock down the apps that can auto start.

    Homindroid,
    If you use Clockwork Mod for backup/restores, when I finish getting it to "clean room" status, do you want a copy of it?
     
  14. Homindroid

    Homindroid Member

    Joined:
    Apr 23, 2010
    Messages:
    32
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Virginia, near the Beach
    Ratings:
    +0
    Skull_One,

    Definately! I have already backup'd my the apps I need for benchmarks to my sd card and bought the "autostarts" app. One though, If I do not activate or sign in, how do I restore those apps from the sd card? Do I push a file manager through ADB? Her is the apps I plan on using;

    • Benchmark PI
    • CPU Benchmark
    • Linpack
    • Quadrant
    • autostarts
    • astro/root explorer (maybe?)
    • Quadrant
    Ofcourse, if you think some of these are redundant or have another suggestion, I'm game.

    I really can't wait to get started with the new "Clean Room" Sapphire! I really appreciate all the help!

    I'm going back to work, teaching till 4. I'll check this thread around 3pm.

    Dustin
     
  15. Skull One

    Skull One Member

    Joined:
    Mar 11, 2010
    Messages:
    759
    Likes Received:
    6
    Trophy Points:
    18
    Ratings:
    +6
    I push Root Explorer right off the bat. Then I install autostarts, SetCPU, Battery Graph, drocap2, OSMonitor and aLogcat. With that baseline installed, I can then kill off the remaining apps that aren't needed for booting, kill the auto launch stuff and then with OSmonitor and aLogcat confirm that the phone is nearly brain dead ;)
     
Search tags for this page

benmark pi apk