If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > MySQL > Query Optimization help

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 03-14-06, 13:44
RogerWilco RogerWilco is offline
Registered User
 
Join Date: Oct 2003
Posts: 268
Query Optimization help

I'm doing a query that is like this:

Code:
SELECT *
FROM A
INNER JOIN B ON (A.Value LIKE CONCAT('%', CONCAT(B.Value, '%')))
Table A has ~25K rows
Table B has ~50K rows.

This query ran for an hour before I killed it. I know the query is inherently slow and won't use indexes, but I need to get better performance. A few minutes is acceptable; over an hour isn't.

Doing an EXPLAIN shows two "all" joins. I know that's bad but I don't see a way around it.

Any ideas?
Reply With Quote
  #2 (permalink)  
Old 03-14-06, 14:21
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,534
you really don't want me to say "we told you to redesign your table" do you?

okay, i won't

__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
  #3 (permalink)  
Old 03-14-06, 14:24
RogerWilco RogerWilco is offline
Registered User
 
Join Date: Oct 2003
Posts: 268
Quote:
Originally Posted by r937
you really don't want me to say "we told you to redesign your table" do you?

okay, i won't

My table design isn't set in stone. I'd love to hear your suggestion.

FYI, this is mostly a one time operation that an administrator will paste into Query Browser (or whatever) and manually run; this isn't something that an end user could trigger via application use.
Reply With Quote
  #4 (permalink)  
Old 03-14-06, 14:28
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,534
assuming you want to do something like finding the word "Foo" in a longer string like "The Inimitable Foo Company", write a script to parse the table A values into multiple rows, and then you can use an index
__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
  #5 (permalink)  
Old 03-14-06, 15:36
RogerWilco RogerWilco is offline
Registered User
 
Join Date: Oct 2003
Posts: 268
Quote:
Originally Posted by r937
assuming you want to do something like finding the word "Foo" in a longer string like "The Inimitable Foo Company", write a script to parse the table A values into multiple rows, and then you can use an index
You mean split a row like "The Inimitable Foo Company" into every possible substring? That seems like overkill and very inefficient.

If I wrote an custom app in Java to do this with the same data set, it would run in no more than a few seconds. I know MySQL is very general purpose and can't meet every special use, but still, there must be a way to get reasonable performance out of these scenarios.
Reply With Quote
  #6 (permalink)  
Old 03-14-06, 15:50
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,534
i understand, but it's not my data, and i don't want to guess at what the best way would be to improve the joins

all i can say is, you may have to do it without indexes, eh
__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On