You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have an instance where I cannot set the database_url env vars (TDDium insists on using sockets). Would it be feasible for QC to fallback to use the ActiveRecord::Base.connection if there is one if the db_url won't resolve?
The only issue I can see at the moment is that the execute method will need changing to suit the different APIs
This is a good issue. I have been thinking about how we could pass connections to QC. Is there an easy way to get an instance of PGconn from ActiveRecord?
It seems that ActiveRecord::Base.connection.raw_connection might get us where we need to go. It seems to return a PG::Connection which is probably a good start.
The following seems to work in my Rails app in config/initializers/queue_classic.rb:
require 'queue_classic'
module QC
module Conn
def self.connection=(connection)
unless connection.instance_of? PG::Connection
raise(
ArgumentError,
"connection must be an instance of PG::Connection, but was #{connection.class}"
)
end
@connection = connection
end
end
end
QC::Conn.connection = ActiveRecord::Base.connection.raw_connection
Am happy to send a pull request based on this, let me know.
Not sure which approach works better here now having see @malclocke's version - however, knowing how hard it was to get the AR connection in the right way (in terms of figuring it out) I feel that falling back to a predicted connection location might be better.
I don't think this is the right approach. I have the same problem, but for me it doesn't work because if I assign the connection in an initializer it might already be teared down/not pointing to the current connecton, when you try to access it.
I think a better approach would be to be able to specify a block that is yielded inside QC::Conn.connect to aquire the connection.
For now I'm solving this problem by overriding connect from an initializer:
@fbuenemann but with your approach, queue_classic must know about ActiveRecord. Can you not delay the handoff of the active_record connection to queue_classic until you are sure that you have the right connection?
Well, my idea was to specify a block that is yielded inside QC, so the content of this block don't matter, as long a it yields a proper connection object.
Setting the connection object at a later point instead of in a initializer would probably work as well.
One thing I don't like about both cases, is that in case of an exception QC tears down the connection it didn't establish itself, which might be bad. For now I patched the disconnect method so it just resets @connection to nil.
I think there's a lot of duplication around connection handling already. There's connect and connection= and we would add yet another connection_proc=. I think a cleaner approach would be to have connect always relying on a proc to establish its connection and the default implementation would parse the database URI for building a connection. Then we could use connection= to pass in a proc that returns a connection. For backwards compatibility connection= could test wheter its param is a connection or a proc.
We would still need to tackle the disconnection problem though, because QC shouldn't be allowed to teardown a passed in connection. I'm not sure what's the best way to handle that.
@joallard even though I merged #134 without it it's still on my todo list. I'm working on a connection pool, which is based on AR so that you should have zero-config out of the box when running within a Rails app.
Establishing my own connection didn't seem to help (gave a different error), although setting QC_DATABASE_URL did. I think @felixbuenemann's solution is better. @senny, I look forward to the ability to pass block to QC::Conn.
@ericboehs using the AR connection is not supported yet. QC currently does not establish a new connection once it gets closed so you will eventually run into exactly the error you are describing. There are some internal changes happening and using the AR-connection by default in a Rails application is still on my todo list.
I recommend setting QC_DATABASE_URL in the meantime.
Thanks for the recommendation. I wrapped my monkey patch with if Rails.env.development? || Rails.env.test?. Setting QC_DATABASE_URL caused rake db:drop to stop working. Not sure why but I'll just wait for your connection pooling stuff.
My app seems to be working in production as I'm using Heroku which sets DATABASE_URL by default.
Which works fine since DATABASE_URL is always set in our production envs and our dev/test envs use localhost with the current user and no password (we set pg to allow logins on loopback only).
Activity
ryansmith3136 commentedon Aug 22, 2012
This is a good issue. I have been thinking about how we could pass connections to QC. Is there an easy way to get an instance of
PGconnfrom ActiveRecord?neilmiddleton commentedon Aug 22, 2012
It seems that
ActiveRecord::Base.connection.raw_connectionmight get us where we need to go. It seems to return aPG::Connectionwhich is probably a good start.malclocke commentedon Aug 23, 2012
The following seems to work in my Rails app in
config/initializers/queue_classic.rb:Am happy to send a pull request based on this, let me know.
Allow setting of QC::Conn.connection (closes QueueClassic#96)
malclocke commentedon Aug 23, 2012
Realised that it's easier to just send a PR, rather than ask if I should send a PR :). Feel free to reject if it's the wrong approach.
neilmiddleton commentedon Aug 23, 2012
Not sure which approach works better here now having see @malclocke's version - however, knowing how hard it was to get the AR connection in the right way (in terms of figuring it out) I feel that falling back to a predicted connection location might be better.
Merge pull request #97 from malclocke/allow_explicit_setting_of_conne…
felixbuenemann commentedon Nov 23, 2012
I don't think this is the right approach. I have the same problem, but for me it doesn't work because if I assign the connection in an initializer it might already be teared down/not pointing to the current connecton, when you try to access it.
I think a better approach would be to be able to specify a block that is yielded inside QC::Conn.connect to aquire the connection.
For now I'm solving this problem by overriding
connectfrom an initializer:ryansmith3136 commentedon Dec 3, 2012
@fbuenemann but with your approach, queue_classic must know about ActiveRecord. Can you not delay the handoff of the active_record connection to queue_classic until you are sure that you have the right connection?
felixbuenemann commentedon Dec 3, 2012
Well, my idea was to specify a block that is yielded inside QC, so the content of this block don't matter, as long a it yields a proper connection object.
Setting the connection object at a later point instead of in a initializer would probably work as well.
One thing I don't like about both cases, is that in case of an exception QC tears down the connection it didn't establish itself, which might be bad. For now I patched the disconnect method so it just resets
@connectionto nil.malclocke commentedon Jun 5, 2013
@felixbuenemann I've just hit the problem you are describing above where we have a reconnect happening after fork in our production set-up.
I think the idea of setting a connection proc is a good one, e.g. something like:
... and that proc will be used by
QC::Conn.connectif provided, otherwise the default connection code is called.Would be worth integrating this with #134 too I think.
I don't see any other way around this with our current production setup (unicorn), other than using
DATABASE_URL.felixbuenemann commentedon Jun 5, 2013
I think there's a lot of duplication around connection handling already. There's
connectandconnection=and we would add yet anotherconnection_proc=. I think a cleaner approach would be to haveconnectalways relying on a proc to establish its connection and the default implementation would parse the database URI for building a connection. Then we could useconnection=to pass in a proc that returns a connection. For backwards compatibilityconnection=could test wheter its param is a connection or a proc.We would still need to tackle the disconnection problem though, because QC shouldn't be allowed to teardown a passed in connection. I'm not sure what's the best way to handle that.
joevandyk commentedon Jun 7, 2013
FWIW, I use sequel and activerecord and queue_classic in the same application. I would love to be able to share the same pg connection for everything.
I'd gladly pay someone to make this work! :)
senny commentedon Jun 9, 2013
This is the only thing missing for #134. I've had a lot of stuff to do lately but I hope to get it done in the upcoming days.
joallard commentedon Aug 16, 2013
Seeing #134 was merged without this, maybe we should reopen the issue? @senny
senny commentedon Aug 17, 2013
@joallard even though I merged #134 without it it's still on my todo list. I'm working on a connection pool, which is based on AR so that you should have zero-config out of the box when running within a Rails app.
joallard commentedon Aug 17, 2013
Ah okay, good to know! Thanks
ericboehs commentedon Sep 9, 2013
I just ran into PG::Error: connection is closed in my integration tests. I fixed it with @felixbuenemann's patch.
Establishing my own connection didn't seem to help (gave a different error), although setting
QC_DATABASE_URLdid. I think @felixbuenemann's solution is better. @senny, I look forward to the ability to pass block to QC::Conn.senny commentedon Sep 10, 2013
@ericboehs using the AR connection is not supported yet. QC currently does not establish a new connection once it gets closed so you will eventually run into exactly the error you are describing. There are some internal changes happening and using the AR-connection by default in a Rails application is still on my todo list.
I recommend setting
QC_DATABASE_URLin the meantime.ericboehs commentedon Sep 10, 2013
Thanks for the recommendation. I wrapped my monkey patch with
if Rails.env.development? || Rails.env.test?. SettingQC_DATABASE_URLcausedrake db:dropto stop working. Not sure why but I'll just wait for your connection pooling stuff.My app seems to be working in production as I'm using Heroku which sets DATABASE_URL by default.
joallard commentedon Sep 10, 2013
My workaround was to simply use the
rails-database-urlgemericboehs commentedon Sep 11, 2013
I saw that. Oddly it caused my integration tests to indefinitely stall. ¯_(ツ)_/¯
A team member suggested we just do
Which works fine since
DATABASE_URLis always set in our production envs and our dev/test envs use localhost with the current user and no password (we set pg to allow logins on loopback only).