Feed item generation is slow
Reported by Michael Hartl | May 23rd, 2008 @ 06:15 PM | in Foundations
Making all the activity feed items for a user with many contacts (such as Michael Hartl at Insoshi) is really slow. (Among other things, this makes registration slow, since new people get connected to the admin by default, and that connection goes into the admin's feed.)
The culprit is the code in lib/activity_logger.rb:
def add_activities(options = {})
person = options[:person]
include_person = options[:include_person]
activity = options[:activity] ||
Activity.create!(:item => options[:item], :person => person)
person.contacts.each do |c|
# Prevent duplicate entries in the feed.
c.activities << activity unless c.activities.include?(activity)
end
if include_person
person.activities << activity unless person.activities.include?(activity)
end
end
I suspect there's some sort of awful O(n2) explosion here. We need to fix it.
Comments and changes to this ticket
-
Michael Hartl June 17th, 2008 @ 04:41 PM
- → Assigned user changed from “” to “Michael Hartl”
- → Milestone changed from “” to “Foundations”
-
Michael Hartl June 17th, 2008 @ 05:23 PM
This naturally divides into two steps:
- Peel off into an asynchronous process
- Profile and optimize
(1) will solve the bottleneck as far as the webapp goes, but the algorithm is probably a ticking time bomb so doing (2) is de rigueur.
-
Michael Hartl July 29th, 2008 @ 08:59 AM
- → State changed from “new” to “open”
- → Tag changed from “” to “enhancement performance”
-
Michael Hartl July 29th, 2008 @ 11:55 PM
The cause of the slowness was all the separate instantiations of the Feed object and the one-by-one insertions. The fix was to consolidate the insert into one statement. This reduces the time to complete an action for my user on the developer site (with ~1000 contacts) from ~20s to < 0.1s, more than two orders of magnitude (!).
The fix is available immediately on the edge branch. The developer site will be updated once we've verified the new code's behavior in production.
-
Michael Hartl July 29th, 2008 @ 05:37 PM
- → State changed from “open” to “resolved”
-
Michael Hartl July 29th, 2008 @ 11:54 PM
The updated feed generation is now running on the developer site, and is available on the public master branch.
-
Michael Hartl July 29th, 2008 @ 11:56 PM
N.B. Because it's so fast, there's no need to make at asynchronous, at least not at this point. We'll cross that bridge when (if) we come to it.
Please Login or create a free account to add a new comment.
You can update this ticket by sending an email to from your email client. (help)
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile »
