Sim: Who are the robot members

Discussion in 'PropertyChat Community & Website' started by Chilliblue, 30th Jun, 2015.

Join Australia's most dynamic and respected property investment community
  1. Chilliblue

    Chilliblue Well-Known Member

    Joined:
    18th Jun, 2015
    Posts:
    1,605
    Location:
    Australia
    I note that under members online, there is a number for robots.

    Just curious.
     
  2. D.T.

    D.T. Specialist Property Manager Business Member

    Joined:
    3rd Jun, 2015
    Posts:
    9,189
    Location:
    Adelaide and Gold Coast
    automated crawlers for search engines that read the website content to add to their databases
     
  3. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,393
    Location:
    Sydney
    Yup, as DT mentioned - they are automated web crawlers.

    At the moment, we have Google indexing the site for their search engine, plus the Adsense crawler (which looks at the page content to try and find relevant adverts to be displayed on that page).

    2015-06-30_14-05-34.png
     
  4. Chilliblue

    Chilliblue Well-Known Member

    Joined:
    18th Jun, 2015
    Posts:
    1,605
    Location:
    Australia
    Thank you
     
  5. bob shovel

    bob shovel Well-Known Member

    Joined:
    18th Jun, 2015
    Posts:
    6,935
    Location:
    Lower Blue Mountains
    that sounds very creepy and futuristic!
    so is it a program scanning all the words and content?? then to be used to help their search engines perform?? is this continuously happening on all web pages out there or based on a visits to what they are then interested in scanning?
    can you not allow it? or is that when you wont come up in search results?

    i dont know a whole lot about the interweb but isnt that more concerning than the government keeping data?
     
  6. legallyblonde

    legallyblonde Well-Known Member

    Joined:
    22nd Jun, 2015
    Posts:
    858
    Location:
    TAS
    Without it people would be far less likely to stumble upon this forum =) I found SS by googling a property related question on CGT and I have never looked back!
     
  7. D.T.

    D.T. Specialist Property Manager Business Member

    Joined:
    3rd Jun, 2015
    Posts:
    9,189
    Location:
    Adelaide and Gold Coast
    Yep, yep, yep and yep.
     
  8. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,393
    Location:
    Sydney
    Dude - do you even internet? :p

    The entire internet works because of Google. Indeed, there are some novice users who believe that Google is the internet.

    Yes, you can stop your site from being indexed by Google - there is a standard system for blocking robots - but it does rely on their cooperation (the big search engines do honour these systems, so it does work for the most part).

    The problem is the smaller bots which are not related to the large search engines - some of them don't honour the standard blocking approaches and will continue to index your website regardless - however, you could try blocking based on user agent or IP address or something if you really wanted them to stop.

    The only sure way to block search engines and other bots is to put your content behind some type of authentication system (ie require a password to access your content). Private websites like that don't get crawled or indexed.
     
  9. bob shovel

    bob shovel Well-Known Member

    Joined:
    18th Jun, 2015
    Posts:
    6,935
    Location:
    Lower Blue Mountains
    wow! ive lead sheltered life!

    i have learnt something for the day... off to bed i go i guess, my work is done

    i knew computers werr a big deal but my year 9 computer killed the love affair. i got really good at quake online that year, thanks to the kids that got it up and running. circa '98. still love thst game
     
  10. Propertunity

    Propertunity Well-Known Member

    Joined:
    19th Jun, 2015
    Posts:
    3,476
    Location:
    NSW
    They have the internet on computers now.
     
    bob shovel likes this.
  11. Hodge

    Hodge Well-Known Member

    Joined:
    18th Jun, 2015
    Posts:
    674
    Location:
    Melbourne
    Bit off topic but what would happen to the Internet if Google was to shut up shop?
     
  12. Simon Hampel

    Simon Hampel Founder Staff Member

    Joined:
    3rd Jun, 2015
    Posts:
    12,393
    Location:
    Sydney
    It's not so much the internet - it's the number of businesses that rely on Google to direct traffic to them that would suffer.

    We'd be forced to go back to the Yellow Pages to find local businesses :eek: