If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > MySQL > problem with group by

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 11-20-04, 10:35
maven maven is offline
Registered User
 
Join Date: Nov 2004
Posts: 17
problem with group by

Hi I would really appreciate some help onundefined this one as I've tried various methods and can't get this to

work on

TABLE 1 (LOGDATA)

ip service hit

1.1.1.1 UDP 200
1.1.1.1 UDP 100
1.1.1.1 TCP 300
1.1.1.1 HTTP 100
2.2.2.2 UDP 200
2.2.2.2 TCP 400

I GOT AN REQUIREMENT TO COPY DATA FROM ABOVE TABLE1(LOGDATA) TO TABLE2 (RANK).
AND I AM LOOKING OUTPUT LIKE BELOW.

TABLE 2 (RANK)

ip service hit RANK1 RANK2

1.1.1.1 UDP 300 1 1
1.1.1.1 TCP 200 1 2
1.1.1.1 HTTP 100 1 3
2.2.2.2 UDP 200 2 2
2.2.2.2 TCP 400 2 1

IT SHOULD UPDATE RANK1 LIKE GROUP BY IP AND ORDER HIT, HIGEST HITS IP WILL BE RANK1.
AND
RANK2 SHOULD BE LIKE WHERE IP=IP GROUP BY SERVICE ORDER BY HIT.

Maybe it can be done with joins but I can't get my head around it.

I would greatly appreciate it if someone could point me in the right direction.


THANKS IN ADVANCE.

MAVEN
Reply With Quote
  #2 (permalink)  
Old 11-20-04, 12:00
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,534
the easiest way to obtain the rankings is by creating a temp table with an auto_increment column and then inserting the results from a query which has an ORDER BY

do that twice, to 2 different temp tables

then do your insert into the final table by inserting the results of a join
__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
  #3 (permalink)  
Old 11-21-04, 23:49
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,534
Code:
create table ranking1
( rank1 integer not null primary key auto_increment
, ip varchar(15)
, hit integer
);
insert into ranking1 (ip, hit) 
select ip, sum(hit) from table1
group by ip
order by sum(hit) desc
;
create table ranking2
( rank2 integer not null primary key auto_increment
, service varchar(3)
, hit integer
);
insert into ranking2 (service, hit) 
select service, sum(hit) from table1
group by service
order by sum(hit) desc
;
insert into table2
select table1.ip 
     , table1.service 
     , table1.hit
     , ranking1.rank1
     , ranking2.rank2
  from table1
inner
  join ranking1
    on table1.ip
     = ranking1.ip
inner
  join ranking2
    on table1.service
     = ranking2.service
order 
    by ranking1.rank1
     , ranking2.rank2
;
__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
  #4 (permalink)  
Old 11-22-04, 00:20
maven maven is offline
Registered User
 
Join Date: Nov 2004
Posts: 17
please guide me to proceed further

Hi,

I have tried with below query. I am not sure how to get help of temp tables in below query. Please guide me how to proceed further. Tell me, the way i am doing is correct way or not.

insert into rank1 (ip,service,hit)
SELECT
Logdata.ip as ip,
Logdata.service,
sum(Logdata.hit) as hit,
from
(
SELECT
logdateid,
ip,
service,
sum(hit) as hit
from
Logdata
group by ip
order by hit
desc limit 100
) as aac
RIGHT JOIN
Logdata
ON aac.ip=Logdata.ip
group by
Logdata.ip
order by aac.hit desc

Thanks for your help.
Reply With Quote
  #5 (permalink)  
Old 11-22-04, 19:41
maven maven is offline
Registered User
 
Join Date: Nov 2004
Posts: 17
not getting performance with temp tables.

Hi,

Thanks for your response. I have tried with creating 2 temp tables (ranking1, ranking2). But its taking toomuch time to execute query, when there are more records in DB. At present I have tested with around 637434 records in table1. Please let me know is there any other way to get better performance?

Thanks.
Reply With Quote
  #6 (permalink)  
Old 11-22-04, 21:59
guelphdad guelphdad is offline
Registered User
 
Join Date: Mar 2004
Posts: 440
yes, index your tables on any columns that are used in a join.
Reply With Quote
  #7 (permalink)  
Old 11-23-04, 05:20
maven maven is offline
Registered User
 
Join Date: Nov 2004
Posts: 17
rank2 problem ....

hi,

In above query rank2 should be given for one particuar IP.

Result should be like below.

IP - SERVICE - HIT - RANK1 - RANK2

1.1.1.1 - TCP - 300 - 1 - 2
1.1.1.1 - UDP - 400 - 1 - 1
2.2.2.2 - TCP - 150 - 2 - 1
2.2.2.2 - UDP - 100 - 2 - 2

Hits IP 1.1.1.1 using UDP are 400 so rank2 should be 1
and TCP 300, rank2 2.
Query which you gave is grouping on total records.

Please guide how to change this, to get better performance.

Thanks.
Reply With Quote
  #8 (permalink)  
Old 11-23-04, 08:15
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,534
ah, i see, rank2 has to start over at 1 for each new ip

okay, where i defined and populate table ranking2, try replacing that part of the code with this --
Code:
create table ranking2
( ip varchar(15)
, rank2 integer not null auto_increment
, primary key (ip,rank2)
, service varchar(3)
, hit integer
);
insert into ranking2 (ip,service,hit) 
select ip, service, hit from table1
order by 1p, hit desc
and have a look at those results to make sure they're working before proceeding to adjust the join that brings them together with the ranking1 results
__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
  #9 (permalink)  
Old 11-23-04, 11:24
maven maven is offline
Registered User
 
Join Date: Nov 2004
Posts: 17
generating duplicate records in table 2.

Hi,

Thanks for your help. Below query is generating duplicate records in table2 (I think we have to do group by serive). Just I want to make sure, I am following correctly. Please check this once and correct me if I am wrong.

insert into table2(ip,service,hit,rank1,rank2)
select table1.ip
, table1.service
, table1.hit
, ranking1.rank1
, ranking2.rank2
from table1
inner
join ranking1
on table1.ip
= ranking1.ip
inner
join ranking2
on table1.service
= ranking2.service
order
by ranking1.rank1
, ranking2.rank2;


And I want to take only top 100(group by ip) records from table1. Where can I include this requirement in this.
Sorry to giving you trouble, by asking again.

Thanks for your help.
Reply With Quote
  #10 (permalink)  
Old 11-23-04, 11:33
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,534
inner
join ranking2
on table1.ip
= ranking2.ip
and table1.service
= ranking2.service
__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
  #11 (permalink)  
Old 11-23-04, 11:40
maven maven is offline
Registered User
 
Join Date: Nov 2004
Posts: 17
duplicate records

Hi,

I have tried before by adding above condition, but its still giving duplicate records in table2.

Thanks.
Reply With Quote
  #12 (permalink)  
Old 11-23-04, 12:12
maven maven is offline
Registered User
 
Join Date: Nov 2004
Posts: 17
hi,

And also it has to group services, if particular ip has same service then it should sum like below

Table 1 :

ip - service - hit
1.1.1.1-UDP-200
1.1.1.1-UDP-100
1.1.1.1-TCP-200
2.2.2.2-UDP-200


Table 2 will be like :

ip - service - hit - rank1 - rank2
1.1.1.1-UDP-300 - 1 - 1
1.1.1.1-TCP-200 - 1 - 2
2.2.2.2-UDP-200 - 2 - 1

sorry, I think I have not given inputs properly in my initial mail.

Thanks.
Reply With Quote
  #13 (permalink)  
Old 11-23-04, 12:13
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,534
please show some rows from your ranking2 table
__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
  #14 (permalink)  
Old 11-23-04, 12:24
maven maven is offline
Registered User
 
Join Date: Nov 2004
Posts: 17
Quote:
Originally Posted by maven
hi,

And also it has to group services, if particular ip has same service then it should sum like below

Table 1 :

ip - service - hit
1.1.1.1-UDP-200
1.1.1.1-UDP-100
1.1.1.1-TCP-200
2.2.2.2-UDP-200


Table 2 will be like :

ip - service - hit - rank1 - rank2
1.1.1.1-UDP-300 - 1 - 1
1.1.1.1-TCP-200 - 1 - 2
2.2.2.2-UDP-200 - 2 - 1

sorry, I think I have not given inputs properly in my initial mail.

Thanks.
Hi,

TABLE 1
ip-service-hit
1.1.1.1-UDP-300
1.1.1.1-UDP-100
1.1.1.1-TCP-200
1.1.1.1-HTTP-100
2.2.2.2-UDP-200
2.2.2.2-TCP-400


TABLE 2

IP-SERVICE-HIT-RANK1-RANK2
2.2.2.2-TCP-400-1-1
2.2.2.2-UDP-200-1-2
1.1.1.1-UDP-200-2-1
1.1.1.1-UDP-100-2-1
1.1.1.1-TCP-200-2-2
1.1.1.1-UDP-200-2-3
1.1.1.1-UDP-100-2-3

EXPECTED RESULT :

IP-SERVICE-HIT-RANK1-RANK2
1.1.1.1-UDP-400-1-1
1.1.1.1-TCP-200-1-2
1.1.1.1-HTTP-100-1-3
2.2.2.2-TCP-400-2-1
2.2.2.2-UDP-200-2-2

Thanks.
Reply With Quote
  #15 (permalink)  
Old 11-23-04, 12:31
maven maven is offline
Registered User
 
Join Date: Nov 2004
Posts: 17
table2 data

Hi,

TABLE 1
ip-service-hit
1.1.1.1-UDP-300
1.1.1.1-UDP-100
1.1.1.1-TCP-200
1.1.1.1-HTTP-100
2.2.2.2-UDP-200
2.2.2.2-TCP-400


TABLE 2 (present query result)

IP-SERVICE-HIT-RANK1-RANK2
2.2.2.2-TCP-400-1-1
2.2.2.2-UDP-200-1-2
1.1.1.1-UDP-200-2-1
1.1.1.1-UDP-100-2-1
1.1.1.1-TCP-200-2-2
1.1.1.1-UDP-200-2-3
1.1.1.1-UDP-100-2-3

table2 EXPECTED RESULT :

IP-SERVICE-HIT-RANK1-RANK2
1.1.1.1-UDP-400-1-1
1.1.1.1-TCP-200-1-2
1.1.1.1-HTTP-100-1-3
2.2.2.2-TCP-400-2-1
2.2.2.2-UDP-200-2-2

Thanks.
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On