ruby on rails - How to let the Facebook scraper into dynamic, authenticated pages -

ruby on rails - How to let the Facebook scraper into dynamic, authenticated pages -

i have social network requires authentication , email verification before user can enter. 1 time inside, users can see content friends. simple, if doesn't sound it. here authenticate before filter:

def authenticate if logged_in? redirect_to authentication_url if current_user.account_disabled else redirect_to root_url end end

the problem have letting facebook scraper in meta tags of dynamic pages. read can allow facebook's user agent non public pages, isn't pages protected in robots.txt file? i'm not experienced scrapers certainly need cookie , enabled business relationship scrape dynamic info on site? i'm not sure how write method allow scraper in or write it.

i'll though generating token securerandom.urlsafe_base64 scraper , making exception on blank page (with meta data) shouldn't accessable regular users, technically wouldn't safe, considering if looked @ right js file (for url reference in open graph action post) , meta tags protected user data. thought doesn't seem close correct...

any ideas?

as long content has unique urls each user sees (normally protected login filter), can allow access checking source ip or user agent match facebook scraper.

however, social sites, using same urls homecoming customized contents rendered logged in user. inherently unscrapable - because there different version of '/profile' each user.

ruby-on-rails ruby facebook facebook-opengraph scraper