Details

    • Type: Task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: node-cubrid 3.0.0
    • Fix Version/s: node-cubrid 3.0.0
    • Component/s: Node.js
    • Labels:
      None

      Description

      As a basis for performance benchmarking I took https://github.com/Sannis/node-mysql-benchmarks which provides performance benchmarking for various types of Node.js drivers for MySQL.

      I added a script for CUBRID which performs the same operations. The results are as follows:

      {code}
      Server: Ubuntu 12.04.2 x64

      $ uname -a
      Linux ubuntu 3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

      $ mysql --version
      mysql Ver 14.14 Distrib 5.5.31, for debian-linux-gnu (x86_64) using readline 6.2

      $ cubrid --version
      cubrid (CUBRID utilities)
      CUBRID 9.1 (9.1.0.0212) (64bit release build for linux_gnu) (Mar 14 2013 00:16:33)
      Copyright (C) 2008 Search Solution Corporation. All rights reserved by Search Solution.

      Client: Mac OS X 10.8.4

      $ node --version
      v0.10.9

      $ make benchmark
      rm -rf ./build
      ./benchmarks.js
      Benchmarking with factor 1...
      Benchmarking 'mysql'... Done.
      Cooldown...
      Benchmarking 'cubrid'... Done.
      Cooldown...

      Results:
      ---------------------------------------------------------------------------------+
      | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects |
      ---------------------------------------------------------------------------------+
      | mysql | V | V | 0.103 | 638442 | - | 241 | 86093 |
      ---------------------------------------------------------------------------------+
      | cubrid | V | | 1.026 | 659800 | - | 118 | 33287 |
      ---------------------------------------------------------------------------------+
      Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).


      $ node benchmarks -f 10
      Benchmarking with factor 10...
      Benchmarking 'mysql'... Done.
      Cooldown...
      Benchmarking 'cubrid'... Done.
      Cooldown...

      Results:
      ---------------------------------------------------------------------------------+
      | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects |
      ---------------------------------------------------------------------------------+
      | mysql | V | V | 0.07 | 599735 | - | 322 | 95232 |
      ---------------------------------------------------------------------------------+
      | cubrid | V | | 1.068 | 729820 | - | 133 | 42037 |
      ---------------------------------------------------------------------------------+
      Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).

      $ node benchmarks -f 10
      Benchmarking with factor 10...
      Benchmarking 'cubrid'... Done.
      Cooldown...
      Benchmarking 'mysql'... Done.
      Cooldown...

      Results:
      ---------------------------------------------------------------------------------+
      | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects |
      ---------------------------------------------------------------------------------+
      | cubrid | V | | 1.151 | 655652 | - | 129 | 45332 |
      ---------------------------------------------------------------------------------+
      | mysql | V | V | 0.067 | 578854 | - | 281 | 112914 |
      ---------------------------------------------------------------------------------+
      Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).

      $ node benchmarks -f 10
      Benchmarking with factor 10...
      Benchmarking 'cubrid'... Done.
      Cooldown...
      Benchmarking 'mysql'... Done.
      Cooldown...

      Results:
      ---------------------------------------------------------------------------------+
      | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects |
      ---------------------------------------------------------------------------------+
      | cubrid | V | | 1.037 | 715812 | - | 123 | 44907 |
      ---------------------------------------------------------------------------------+
      | mysql | V | V | 0.063 | 593422 | - | 333 | 74429 |
      ---------------------------------------------------------------------------------+
      Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).

      $ node benchmarks -f 10
      Benchmarking with factor 10...
      Benchmarking 'cubrid'... Done.
      Cooldown...
      Benchmarking 'mysql'... Done.
      Cooldown...

      Results:
      ---------------------------------------------------------------------------------+
      | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects |
      ---------------------------------------------------------------------------------+
      | cubrid | V | | 0.106 | 677260 | - | 121 | 36548 |
      ---------------------------------------------------------------------------------+
      | mysql | V | V | 0.318 | 576863 | - | 280 | 102311 |
      ---------------------------------------------------------------------------------+
      Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).{code}

      In average CUBRID tests show slower performance in comparison with node-mysql module: initialization (connectin + drop table + create table - 10x), inserts (2x), selects (2.5x). The initialization time was dramatically improved (actually node-cubrid show 2x faster performance than node-mysql) by replacing queued execution by callback execution.

      from:

      {code}
      conn.addNonQuery("DROP TABLE IF EXISTS " + cfg.test_table);

      conn.addNonQuery(cfg.cubrid.create_table_query, function (err) {
      if (err) { errorHandler(err); } else { results.init = helper.roundWithPrecision(helper.hrtimeDeltaInSeconds(start_hrtime), 3); escapeBenchmark(results, callback, cfg, benchmark); }
      });{code}

      to:

      {code}
      // Implicitly will create a connection.
      conn.execute("DROP TABLE IF EXISTS " + cfg.test_table, function (err) {
      if (err) { errorHandler(err); } else {
      conn.execute(cfg.cubrid.create_table_query, function (err) {
      if (err) { errorHandler(err); } else { results.init = helper.roundWithPrecision(helper.hrtimeDeltaInSeconds(start_hrtime), 3); escapeBenchmark(results, callback, cfg, benchmark); }
      });
      }
      });{code}

      It is a known issue (APIS-529) that queued queries are executed way slower (4x) than direct callback execution.

      Though these performance results don't reveal the cause why INSERT/SELECT operations are slow in node-cubrid, it is definite that either the server or Node.js module implementation can affect the performance.

      There are many things we can improve in node-cubrid alone to provide a better performance. The first thing to look at is the queueing mechanism which is 4x slower than executing queries directly.

        Activity

        Hide
        kadishmal Esen Sagynov added a comment - - edited

        More results. This time running tests for each driver separately after cold restart: new VM, install DB Server, run tests, delete VM. CPU 1, RAM: 1G.

        {code}
        node-cubrid-performance-benchmarks nbp$ node benchmarks -f 10
        Benchmarking with factor 10...
        Benchmarking 'node-cubrid-2.0.2'... Done.
        Cooldown...

        Results:
        ---------------------------------------------------------------------------------+
        | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects |
        ---------------------------------------------------------------------------------+
        | node-cubrid-2.0.2 | V | | 0.061 | 843274 | - | 509 | 172415 |
        ---------------------------------------------------------------------------------+
        Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).{code} {code}
        node-cubrid-performance-benchmarks nbp$ node benchmarks -f 10
        Benchmarking with factor 10...
        Benchmarking 'node-cubrid-2.1.0'... Done.
        Cooldown...

        Results:
        ---------------------------------------------------------------------------------+
        | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects |
        ---------------------------------------------------------------------------------+
        | node-cubrid-2.1.0 | V | | 0.102 | 773770 | - | 488 | 113698 |
        ---------------------------------------------------------------------------------+
        Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).{code} {code}
        node-cubrid-performance-benchmarks nbp$ node benchmarks -f 10
        Benchmarking with factor 10...
        Benchmarking 'mysql'... Done.
        Cooldown...

        Results:
        ---------------------------------------------------------------------------------+
        | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects |
        ---------------------------------------------------------------------------------+
        | mysql | V | | 0.031 | 733827 | - | 1844 | 104693 |
        ---------------------------------------------------------------------------------+
        Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).{code} {code}
        node-cubrid-performance-benchmarks nbp$ node benchmarks -f 10
        Benchmarking with factor 10...
        Benchmarking 'mysql typecast'... Done.
        Cooldown...

        Results:
        ---------------------------------------------------------------------------------+
        | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects |
        ---------------------------------------------------------------------------------+
        | mysql typecast | V | V | 0.026 | 692933 | - | 1812 | 129753 |
        ---------------------------------------------------------------------------------+
        Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).{code}
        Show
        kadishmal Esen Sagynov added a comment - - edited More results. This time running tests for each driver separately after cold restart: new VM, install DB Server, run tests, delete VM. CPU 1, RAM: 1G. {code} node-cubrid-performance-benchmarks nbp$ node benchmarks -f 10 Benchmarking with factor 10... Benchmarking 'node-cubrid-2.0.2'... Done. Cooldown... Results: ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects | ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ | node-cubrid-2.0.2 | V | | 0.061 | 843274 | - | 509 | 172415 | ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).{code} {code} node-cubrid-performance-benchmarks nbp$ node benchmarks -f 10 Benchmarking with factor 10... Benchmarking 'node-cubrid-2.1.0'... Done. Cooldown... Results: ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects | ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ | node-cubrid-2.1.0 | V | | 0.102 | 773770 | - | 488 | 113698 | ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).{code} {code} node-cubrid-performance-benchmarks nbp$ node benchmarks -f 10 Benchmarking with factor 10... Benchmarking 'mysql'... Done. Cooldown... Results: ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects | ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ | mysql | V | | 0.031 | 733827 | - | 1844 | 104693 | ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).{code} {code} node-cubrid-performance-benchmarks nbp$ node benchmarks -f 10 Benchmarking with factor 10... Benchmarking 'mysql typecast'... Done. Cooldown... Results: ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ | Name | Async | TypeCast | Init | Escapes | Reconnects | Inserts | Selects | ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ | mysql typecast | V | V | 0.026 | 692933 | - | 1812 | 129753 | ----------------------------- ----- -------- ----- --------- ---------- ------- --------+ Init time in seconds (doesn't matter), other values are operations/rows per second (more is better).{code}

          People

          • Assignee:
            kadishmal Esen Sagynov
            Reporter:
            ovidiu.guta Ovidiu Guta
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0 minutes
              0m
              Logged:
              Time Spent - 1 day, 1 hour
              1d 1h