Results 1 to 3 of 3
  1. #1
    Join Date
    Mar 2010
    Posts
    9

    Unanswered: High concurrent insert and managing millions of data daily

    1.I am having a system which receive data from around 30K gps . Each unit will be inserting concurrently around 1 minute. What tweak should I add to cater for this? I target my ram to be 48Gb with 24 cpus.
    2.Once the data grow should I do partition or leave it . I am worried as the data grow the insert will be slow?
    3.What should I anticipate and handling for heavy traffic?

    Below is one of data structure which will grow into millions daily.
    Code:
    `gdata` ( 
    `alarmTypeID` tinyint(4) NOT NULL DEFAULT '0', 
    `fleetID` smallint(11) NOT NULL, 
    `fleetGroupID` smallint(11) DEFAULT NULL, 
    `fleetSubGroupID` smallint(11) DEFAULT NULL, 
    `deviceID` mediumint(11) NOT NULL, 
    `vehicleID` mediumint(11) NOT NULL, 
    `gDateTime` datetime NOT NULL, 
    `insertDateTime` datetime NOT NULL, 
    `latitude` float NOT NULL, 
    `longitude` float NOT NULL, 
    `speed` smallint(11) NOT NULL,
    (see full text)
    ALTER TABLE `gdata` 
    ADD PRIMARY KEY (`alarmTypeID`,`vehicleID`,`gDateTime`), 
    ADD KEY `gDateTime` (`gDateTime`), 
    ADD KEY `fleetID` (`fleetID`,`vehicleID`,`gDateTime`); 
    COMMIT;

  2. #2
    Join Date
    Jun 2017
    Posts
    7
    Go with ssd hard drives, there is no need to create new partition, sql is designed to manage huge data. If you are inserting only then it should ne be a big problem.

  3. #3
    Join Date
    Mar 2010
    Posts
    9
    Quote Originally Posted by robert4u View Post
    Go with ssd hard drives, there is no need to create new partition, sql is designed to manage huge data. If you are inserting only then it should ne be a big problem.
    Can it handle how many insert per seconds actually? I am using percona mysql. I am not just inserting but I am also doing reading. I am planning to have replication so reading can be done via replication? So when is partition is needed?

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •