If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > MySQL > Performance problem with "in" operator

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 02-26-09, 02:25
naresh_singam naresh_singam is offline
Registered User
 
Join Date: Apr 2008
Posts: 7
Performance problem with "in" operator

Hi all,

I have the following query for which I am unable to retrieve the data when the "account" table has the 1lack+ records. When the data is less I got the result. I checked the query by splitting it I found that the "in" operator is taking time. Can any body help me to tune this query. I need to execute this query from the screen and display the data.

select a.username, count(a.acctidentifier) , sum(a.time), sum(a.input), sum(a.output)
from account a, client c where a.clientid = c.clientid and
a.acctid in (select max(acctid) from account group by acctidentifier)
and date(a.acctrecdate) >= '2009-02-24' and date(a.acctrecdate) <= '2009-02-24' group by a.username

Thanks in Advance
Naresh.
Reply With Quote
  #2 (permalink)  
Old 02-26-09, 04:46
mike_bike_kite mike_bike_kite is offline
vaguely human
 
Join Date: Jun 2007
Location: London
Posts: 2,519
Your query doesn't make sense to me:
  • What are you trying to do in this part?
    Code:
             select max(acctid) 
             from account 
             group by acctidentifier
  • Is this really what you want to put here?
    Code:
    and date(a.acctrecdate) >= '2009-02-24' 
    and date(a.acctrecdate) <= '2009-02-24'
  • Can you explain this part more.
    Quote:
    when the "account" table has the 1lack+ records
  • Using IN is always inefficient but lets find out what you want before we improve things.
Reply With Quote
  #3 (permalink)  
Old 02-26-09, 06:09
healdem healdem is online now
Jaded Developer
 
Join Date: Nov 2004
Location: out on a limb
Posts: 9,262
lakh is Indian for 100,000
see Lakh - Wikipedia, the free encyclopedia for details
__________________
I'd rather be riding my Versys or my Tiger 800 let alone the Norton
Reply With Quote
  #4 (permalink)  
Old 02-26-09, 08:37
naresh_singam naresh_singam is offline
Registered User
 
Join Date: Apr 2008
Posts: 7
Thanks mike_bike_kite for your response,

Coming to your questions

1. In the account table for each "acctidentifier" value there will be multiple "acctid" values out of that I need the latest "acctid". For that I used the "select max(acctid) from account group by acctidentifier".
ex: acctid acctidentifier
----------------------------
1 acc1
2 acc1
3 acc1
4 acc2
5 acc2

In the above lines I need to consider only the Bolded rows.

2. The column "acctrecdate" datatype is timestamp. But I need to compare it with date part only. For that reason I used "and date(a.acctrecdate) >= '2009-02-24' ".

3. The Account table is having more than 100,000 records.

Thanks in Advance,
Naresh.
Reply With Quote
  #5 (permalink)  
Old 02-26-09, 08:47
mike_bike_kite mike_bike_kite is offline
vaguely human
 
Join Date: Jun 2007
Location: London
Posts: 2,519
Quote:
1. In the account table for each "acctidentifier" value there will be multiple "acctid" values out of that I need the latest "acctid". For that I used the "select max(acctid) from account group by acctidentifier".
ex: acctid acctidentifier
----------------------------
1 acc1
2 acc1
3 acc1
4 acc2
5 acc2
Would it be possible to calculate these values before doing the query and then just joining to this new data?

What are the indexes on the account table and roughly how many rows?
Quote:
2. The column "acctrecdate" datatype is timestamp. But I need to compare it with date part only. For that reason I used "and date(a.acctrecdate) >= '2009-02-24' ".
But your testing this date to be >= '2009-02-24' and <= '2009-02-24' . Do you just mean date(a.acctrecdate) = '2009-02-24'.
Reply With Quote
  #6 (permalink)  
Old 02-26-09, 08:51
gvee gvee is offline
www.gvee.co.uk
 
Join Date: Jan 2007
Location: UK
Posts: 10,156
How's this for you?
Code:
SELECT a.username
     , Count(a.acctidentifier
     , Sum(a.time)
     , Sum(a.input)
     , Sum(a.output)
FROM   account As a
 INNER
  JOIN client As c
    ON a.clientid = c.clientid
 INNER
  JOIN (
        SELECT acctidentifier
             , Max(acctid) As max_acctid
        FROM   account
        GROUP
            BY acctidentifier
       ) As x
    ON a.acctidentifier = x.acctidentifier
   AND a.acctid = x.max_acctid
WHERE  Date(a.acctrecdate) = '2009-02-24'
__________________
George
Twitter | Blog
Reply With Quote
  #7 (permalink)  
Old 02-26-09, 09:05
mike_bike_kite mike_bike_kite is offline
vaguely human
 
Join Date: Jun 2007
Location: London
Posts: 2,519
I was getting there George but I just wanted to clear up the thought process in the original SQL first. It's still worth looking at the indexes on the account table though. Also if there is an index on the acctrecdate field then the query still won't use it as you're applying a function to the field ( Date(a.acctrecdate) = '2009-02-24' ).
Reply With Quote
  #8 (permalink)  
Old 02-26-09, 09:12
gvee gvee is offline
www.gvee.co.uk
 
Join Date: Jan 2007
Location: UK
Posts: 10,156
Agreed Mike - the SQL I posted is only part of the solution addressing the IN and no other part.
__________________
George
Twitter | Blog
Reply With Quote
  #9 (permalink)  
Old 02-27-09, 02:16
naresh_singam naresh_singam is offline
Registered User
 
Join Date: Apr 2008
Posts: 7
Thanks gvee and mike_bike_kite,

gvee: The query which u posted is retrieving data. Now I need to validate the data with different scenarios whether that query is retreiving the valid data or not.

mike_bike_kite: For your questions
Code:
Would it be possible to calculate these values before doing the query and then just joining to this new data?
No, I need to give this query in a file. First I will retrieve the qurey string from the file and then I will retreive the data from database.

Code:
What are the indexes on the account table and roughly how many rows?
Currently I don't have any idexes, because this table will be used to insert data more times than retreiving data. But the "acctid" is the primary key in the "account" table.

Thanks,
Naresh.
Reply With Quote
  #10 (permalink)  
Old 02-27-09, 03:25
mike_bike_kite mike_bike_kite is offline
vaguely human
 
Join Date: Jun 2007
Location: London
Posts: 2,519
Does the query provided by George run fast enough for what you need?
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On