by Harold Davis
Build Your Own Advanced Searching Programs
Google's Advanced Features Can Put Your Research On The Fast Track!
Part I: Getting Started with Google Research
Part II: Crafting Queries and Using
Part III: Building Valuable Research Content
Part IV: Building Research Tools
Part V: The Part of Tens
Part VI: Appendixes
 |
| Building Research Tools With Google for Dummies |
Harold Davis is a strategic technology consultant, hands-on computer programmer,
and the author of more than 20 books.
In addition to his work as a writer, Harold has been a technology company
executive, enterprise consultant, software developer, professional photographer,
and a legal researcher.
He has earned a Bachelors Degree in Computer Science and Mathematics
from New York University and a Juris Doctorate from Rutgers Law School,
where he was an editor of the law review.
Harold lives with his wife, Phyllis Davis, who is also an author, and their three
sons — Julian, Nicholas, and Mathew — in the hills of Berkeley, California.
His three sons keep Harold pretty busy, but in his spare time he likes to
garden, hike, and poke around the Internet looking for new and obscure
research resources.
The address for Harold’s personal Web site is www.bearhome.com. He is
a co-founder of Braintique, a site that offers free Web content, located at
www.braintique.com.
Author’s Acknowledgments
Special thanks to Jeff Cogswell, Phyllis Davis, Nicole Haims, Melody Layne,
and Matt Wagner, without whom this book would not have been possible.
Introduction
I can’t think of any simple modifiers that describe Google.
I was going to say that Google is “a complex piece of software.” Then I was
going to describe it as “a research environment with many moving parts.”
And then finally I was going to use the phrase, “a way of life.” Anyway, you
probably already have a pretty good idea of what Google is, even if, like me,
you don’t find Google easy to describe simply.
However you might describe Google, no one doubts that it’s a great tool for
researchers. What you might not know is that significant resources are available
within the Google family that few people know about. For one thing, Google
provides access to its essential functionality with an XML-based SOAP Web
service — called the Google APIs Web service, or simply the Google APIs.
About This Book
This book describes the different aspects of Google and provides the information
you need to successfully
Understand the various Google tools, and how they can help satisfy your research needs
Use Google’s custom search syntax to effectively find information
Determine whether the information you need can be found through
Google (and where to start looking for it if it cannot)
Become a better researcher by learning tips, tricks, and techniques
honed by top research professionals
Learn how Google works, and what mechanisms it uses to respond to search queries
Harness the power of the Google APIs to build your own research tools
If you want to understand how to become a better researcher, or how to use
Google more effectively as a research tool, you’ve picked up the right book.
If you understand software development and are interested in writing applications
that extend the functionality of Google using the Google Web APIs,
this book has the information you need.
What You Shouldn’t Read
I never tell anyone not to read anything (particularly if I’ve written it).
Seriously, folks, this book is intended for a number of different kinds of
readers, including
People interested in using Google more effectively
Those interested in research and the Internet
Webmasters who want to better understand how Google works
(perhaps so they know better how to organize their own Web sites)
Researchers and reference librarians who would like to know how to
better use the tools that Google makes available — and perhaps want
to know how to best help others use those tools
Programmers (or researchers with a programming background) who
would like to use the Google Web APIs to extend and customize the
functionality of Google, and create great research tools
Not of all of these readers (meaning you) have the same interests, needs, or
background knowledge (although, of course, I hope every word I’ve written
will be fascinating and enthralling to each and every one of you!). If you’re
not interested in a topic, just skip it.
I’ve written each chapter of this book so that it stands on its own. However,
there are a few dependencies you should know about:
Chapter 5, which explains advanced Google operators, probably won’t
make as much sense to you if you haven’t already looked at Chapter 4,
which covers the more basic Google operators.
The case studies showing how to create applications with the Google
APIs in Chapters 16–18 expect that you’ve at least had a look at the
introductory materials explaining how the Google APIs Web service
works in Chapters 13–15.
Also, if you’re in a hurry, you can skip sidebars, which offer info that isn’t
crucial (but that is really interesting).
Foolish Assumptions
I assume that you are computer literate, that you know how to use your Web
browser, and that you have fooled around a bit (or even a lot) with Google. I
assume that you possess a deep curiosity about research, and that you have
a general working knowledge of how things work on the Web.
Part IV of this book assumes that you already know basic programming.
Specifically, I assume you know enough C# and Visual Studio .NET to understand
how to use the Google Web APIs to create new search applications.
Unfortunately, there’s simply no space in this book to teach basic programming,
object-oriented programming, or the .NET framework.
To best understand the code examples in this book, I assume that you can
sight read C#, or at least quickly pick it up.
If you want to re-create and run the programs in this book, I assume that you
have Visual Studio .NET up and running, and know your way around it.
You can use the Google APIs to create programs in any language that can work
with SOAP-based Web services — and you’ll find information to help you in
the appendixes if you decide to work with either of two popular languages,
Java or Visual Basic.
May I recommend some of my other books? If you’ve never programmed, and
would like to learn how, I suggest picking up Learn How to Program Using Any
Web Browser (2004, Apress).
If you have some programming under your belt, and want to understand C#
better, please try my Visual C# .Net Programming (2002, Sybex).
A good source of information about Visual Basic .NET is my Visual Basic .Net
for Windows (2003, Peachpit Press).
One of the best ways to learn about the Java language is to open up Sun
Microsystems’s New to Java site (Sun, after all, wrote the Java language).
Its address is http://java.sun.com/learning/new2java/.
I was going to say that Google is “a complex piece of software.” Then I was
going to describe it as “a research environment with many moving parts.”
And then finally I was going to use the phrase, “a way of life.” Anyway, you
probably already have a pretty good idea of what Google is, even if, like me,
you don’t find Google easy to describe simply.
However you might describe Google, no one doubts that it’s a great tool for
researchers. What you might not know is that significant resources are available
within the Google family that few people know about. For one thing, Google
provides access to its essential functionality with an XML-based SOAP Web
service — called the Google APIs Web service, or simply the Google APIs.
About This Book
This book describes the different aspects of Google and provides the information
you need to successfully
Understand the various Google tools, and how they can help satisfy your research needs
Use Google’s custom search syntax to effectively find information
Determine whether the information you need can be found through
Google (and where to start looking for it if it cannot)
Become a better researcher by learning tips, tricks, and techniques
honed by top research professionals
Learn how Google works, and what mechanisms it uses to respond to search queries
Harness the power of the Google APIs to build your own research tools
If you want to understand how to become a better researcher, or how to use
Google more effectively as a research tool, you’ve picked up the right book.
If you understand software development and are interested in writing applications
that extend the functionality of Google using the Google Web APIs,
this book has the information you need.
What You Shouldn’t Read
I never tell anyone not to read anything (particularly if I’ve written it).
Seriously, folks, this book is intended for a number of different kinds of
readers, including
People interested in using Google more effectively
Those interested in research and the Internet
Webmasters who want to better understand how Google works
(perhaps so they know better how to organize their own Web sites)
Researchers and reference librarians who would like to know how to
better use the tools that Google makes available — and perhaps want
to know how to best help others use those tools
Programmers (or researchers with a programming background) who
would like to use the Google Web APIs to extend and customize the
functionality of Google, and create great research tools
Not of all of these readers (meaning you) have the same interests, needs, or
background knowledge (although, of course, I hope every word I’ve written
will be fascinating and enthralling to each and every one of you!). If you’re
not interested in a topic, just skip it.
I’ve written each chapter of this book so that it stands on its own. However,
there are a few dependencies you should know about:
Chapter 5, which explains advanced Google operators, probably won’t
make as much sense to you if you haven’t already looked at Chapter 4,
which covers the more basic Google operators.
The case studies showing how to create applications with the Google
APIs in Chapters 16–18 expect that you’ve at least had a look at the
introductory materials explaining how the Google APIs Web service
works in Chapters 13–15.
Also, if you’re in a hurry, you can skip sidebars, which offer info that isn’t
crucial (but that is really interesting).
Foolish Assumptions
I assume that you are computer literate, that you know how to use your Web
browser, and that you have fooled around a bit (or even a lot) with Google. I
assume that you possess a deep curiosity about research, and that you have
a general working knowledge of how things work on the Web.
Part IV of this book assumes that you already know basic programming.
Specifically, I assume you know enough C# and Visual Studio .NET to understand
how to use the Google Web APIs to create new search applications.
Unfortunately, there’s simply no space in this book to teach basic programming,
object-oriented programming, or the .NET framework.
To best understand the code examples in this book, I assume that you can
sight read C#, or at least quickly pick it up.
If you want to re-create and run the programs in this book, I assume that you
have Visual Studio .NET up and running, and know your way around it.
You can use the Google APIs to create programs in any language that can work
with SOAP-based Web services — and you’ll find information to help you in
the appendixes if you decide to work with either of two popular languages,
Java or Visual Basic.
May I recommend some of my other books? If you’ve never programmed, and
would like to learn how, I suggest picking up Learn How to Program Using Any
Web Browser (2004, Apress).
If you have some programming under your belt, and want to understand C#
better, please try my Visual C# .Net Programming (2002, Sybex).
A good source of information about Visual Basic .NET is my Visual Basic .Net
for Windows (2003, Peachpit Press).
One of the best ways to learn about the Java language is to open up Sun
Microsystems’s New to Java site (Sun, after all, wrote the Java language).
Its address is http://java.sun.com/learning/new2java/.
Table of Contents
Introduction.........
About This Book......
What You Shouldn’t Read.....
Foolish Assumptions .....................
How This Book Is Organized.................
Part I: Getting Started with Google Research .....
Part II: Crafting Queries and Using the Google Research Tools ...
Part III: Building Valuable Research Content ........................
Part IV: Building Research Tools Using the Google APIs....
Part V: The Part of Tens............................................
Part VI: Appendixes........................................
Icons Used in This Book.........................
Conventions Used in This Book .....
Where to Go from Here.......
Part I: Getting Started with Google Research ..................7
Chapter 1: Googling the World . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Searching the Web with Google....................................................................10
Searching using Google’s rules ...........................................................11
Searching for words, not meaning......................................................12
Effectively searching............................................................................12
Refining your search ............................................................................13
Number searches..................................................................................14
Google shortcuts ..................................................................................15
Getting local information.....................................................................17
Mining Google for Information .....................................................................19
Using Google Efficiently with the Google Toolbar .....................................21
Downloading and installing the Google Toolbar ..............................22
Getting to know all the parts of the Google Toolbar........................23
Setting Google Toolbar options ..........................................................25
Chapter 2: Researching with Google and the Web . . . . . . . . . . . . . . . .27
Validating Results on the Web ......................................................................28
Comparing Google with Other Search Engines ..........................................29
Primary source search engines ..........................................................30
Meta-search engines ............................................................................32
Getting a Quick Introduction to Google’s Technology ..............................34
Introducing Search Engine Optimization ....................................................34
Chapter 3: Delivering and Automating Research Results . . . . . . . . . .39
Knowing Your Audience ................................................................................39
Refining Research Questions........................................................................40
Presenting Research Results ........................................................................43
Writing Software That Uses Google .............................................................44
Scripting.................................................................................................44
Introducing the Deskbar......................................................................46
Programming the Deskbar ............................................................................48
Using the Google APIs....................................................................................48
Part II: Crafting Queries and Using
the Google Research Tools............................................51
Chapter 4: Building Simple, Effective Queries . . . . . . . . . . . . . . . . . . . .53
Speeding Up Basic Text Searches ................................................................54
Creating more simple, specific search queries.................................55
Avoiding the I’m-Feeling-Lucky pitfall................................................57
Obtaining Google Desktop Search results.........................................58
Crafting Better Searches ...............................................................................60
Introducing Simple Operators ......................................................................61
Understanding Google operator options ..........................................61
Understanding the AND operator........................................................62
The OR operator....................................................................................62
The inclusion operator ........................................................................63
The exclusion operator .......................................................................64
Refining searches with inclusion and exclusion ..............................64
Quoting Phrases .............................................................................................65
Working with Wildcards ................................................................................67
Using the Advanced Search Page.................................................................68
Understanding Results ..................................................................................70
Using the results...................................................................................72
Cached results ......................................................................................74
Refining your search ............................................................................75
Chapter 5: Achieving Precision with Google Operators . . . . . . . . . . .77
Using the Synonym Operator .......................................................................78
The Related Operator....................................................................................79
Working with Occurrences Operators.........................................................81
Finding your links: allinanchor: and inanchor: operators.......83
Getting into the text: allintext: and intext:..............................86
Finding forms: allintitle: and intitle:.....................................87
Narrowing the search: allinurl: and inurl:................................88
Including and excluding with the occurrences operators ..............89
Getting Definitions .........................................................................................90
Restricting File Types ....................................................................................91
Backtracking with the Link Operator ..........................................................92
Finding a Phone Number...............................................................................93
Combing a Site................................................................................................97
Getting all of Google’s information about a site ...............................97
Using the site: operator....................................................................98
Building a Site Search Box ..........................................................................100
Using ready-made functionality........................................................100
Analyzing Google’s search string .....................................................101
Recreating Google’s search string with JavaScript ........................102
Chapter 6: Using Google Answers to Learn
Google Research Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .105
Taking a Look at Google Answers ..............................................................106
Navigating to Google Answers....................................................................107
Signing Up for Google Answers ..................................................................107
Learning Google Research Techniques .....................................................110
Browsing..............................................................................................111
Searching .............................................................................................113
Writing a Good Query for Google Answers to Research.........................114
Verboten questions ............................................................................115
Posing your question .........................................................................115
Chapter 7: Getting Help from the Google Directory . . . . . . . . . . . . . . .119
Introducing the Open Directory Project ...................................................119
Using the Google Directory.........................................................................121
Opening the Google Directory..........................................................122
Browsing the Google Directory ........................................................124
Searching the Google Directory........................................................126
Making the most of the Google Directory .......................................128
Tips for Working with the Google Directory.............................................130
Google Groups: Extending Usenet .............................................................130
Accessing Google Groups..................................................................131
Browsing groups and dealing with post overload..........................133
Chapter 8: Researching Photographs with Google . . . . . . . . . . . . . . .135
Knowing Google Images Basics ..................................................................135
Understanding how Google Images works......................................135
Accessing the Google Images tool....................................................136
Searching for images..........................................................................137
Understanding image results ............................................................138
Downloading images ..........................................................................139
Using SafeSearch with Images ....................................................................140
Using Advanced Image Search ...................................................................140
Using Pictures in Your Research ................................................................142
Using Google Images to find written information ..........................142
Finding a specific image ....................................................................143
Researching for design ......................................................................144
Chapter 9: Providing Competitive Intelligence:
Researching an Industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147
Finding Out About a Company ...................................................................148
Asking (and answering) crucial questions......................................148
Getting financial information ............................................................150
Your financial research checklist .....................................................152
A sample financial profile ..................................................................154
Understanding an Industry.........................................................................155
Finding out about an industry with Google Directory...................156
Searching the Web with Google to profile an industry..................160
A sample industry profile..................................................................161
Knowing the Players ....................................................................................161
Part III: Building Valuable Research Content ..............165
Chapter 10: Researching Like a Pro . . . . . . . . . . . . . . . . . . . . . . . . . . . .167
Asking Research Questions ........................................................................168
Why ask questions?............................................................................168
Searching hierarchically....................................................................170
Using Reference Interviews ........................................................................171
Interview mechanics and investigative work .................................171
Going through the phases of the reference interview...................172
Why Google Is Not the Web ........................................................................176
Google and other search engines.....................................................176
The “invisible” Web............................................................................177
Other Kinds of Research .............................................................................178
Using a research database ................................................................179
Other online research services ........................................................179
Libraries...............................................................................................180
Public records research.....................................................................181
From Novice to Professional Researcher..................................................182
Ethics and Research ....................................................................................183
Chapter 11: Using Google’s Comparative
Methodology to Your Advantage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .185
The Changing World of Research...............................................................186
Evaluating the Credibility of Information..................................................187
Keeping some basics in mind ...........................................................187
Proactively assessing Web sources..................................................187
Learning How Google Works.......................................................................188
Crawling the Web................................................................................189
Pulling out the keywords...................................................................189
Seeing through hype ..........................................................................190
Searching the doc and index servers...............................................190
The PageRank Algorithm.............................................................................191
Understanding the Implications of PageRank ..........................................193
Utilizing Automated Research Tools .........................................................195
Chapter 12: Packaging and Delivering Research . . . . . . . . . . . . . . . .197
Meeting the Needs of Your Client...............................................................198
Delivering Research Results .......................................................................199
Getting your elevator pitch down pat .............................................200
Preparing an executive summary.....................................................202
Personally Presenting Results ....................................................................204
Effectively using visual information.................................................205
The mechanics of projection preparation.......................................205
Preparing a slideshow........................................................................206
Using E-Mail to Deliver Research Results .................................................208
Using the Web to Present Research Results .............................................209
Sourcing Research Information..................................................................209
When Research Results Are Statistical . . . ...............................................211
Part IV: Building Research Tools
Using the Google APIs ...............................................213
Chapter 13: From Scripting to the Google Web APIs . . . . . . . . . . . . .215
Creating Research Applications with Scripts ...........................................216
The advantages and disadvantages of scripting ............................216
Flowing with your scripts..................................................................217
Creating an automatic translation page ..........................................218
Getting your application coded and ready to go ...........................221
Moving from the Client to the Server ........................................................223
Creating server-side research tools .................................................224
Connecting everything with Web services......................................224
Introducing the Google APIs Web Service.................................................225
Working with the Google APIs ....................................................................227
The APIs are still in beta — should you care? ................................227
Complying with the Terms of Service..............................................228
Investigating infrastructure limitations...........................................229
Querying with the APIs................................................................................229
Getting a page from the cache..........................................................230
Requesting a search...........................................................................230
Understanding search request return values .................................231
Doing a spelling request ....................................................................232
Chapter 14: Downloading a Developer Key . . . . . . . . . . . . . . . . . . . . .233
Getting Your Key ..........................................................................................234
Registering for a Google account .....................................................234
Getting the key by e-mail...................................................................236
Why You Should Get a Key..........................................................................238
Google applications that need an account......................................239
Third-party applications that use a key ..........................................240
Replacing a Lost Key....................................................................................240
Downloading the SDK ..................................................................................241
Chapter 15: Understanding WSDL and Web Services . . . . . . . . . . . .243
Understanding XML Basics.........................................................................244
Getting more eXtensibility for your buck........................................245
Describing XML with schemas .........................................................245
Working with the Google WSDL File...........................................................247
Using the Google WSDL file with Visual Studio .NET .....................247
Using the WSDL file in other languages...........................................249
To HTTP or to SOAP?...................................................................................251
Introducing the SOAP Standard .................................................................251
Calling the Google Web Methods with HTTP............................................252
Using XooMLe to have it both ways ................................................253
Dealing with nonalphanumeric snafus ............................................254
Chapter 16: Building a C# Google Application . . . . . . . . . . . . . . . . . . .257
Creating an Application and Adding a Web Reference............................258
Creating an ASP.NET application......................................................258
Adding a Web reference to an ASP.NET application ......................259
Managing files with Solution Explorer .............................................260
Adding a Web reference to a Windows application .......................261
Working with UDDI from within Visual Studio .NET.......................262
Understanding the generated code and classes ............................263
Programming with the Google APIs ...........................................................265
Making a spelling suggestion ............................................................266
Returning a “lucky” result URL and snippet ...................................268
Understanding the Google SDK Sample ....................................................272
Estimating return results...................................................................273
Finding the size of a cached page ....................................................274
Chapter 17: Creating Your Own Search
Window with the Google APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .275
Constructing the Advanced Search Program ...........................................276
Creating an Advanced Search Window .....................................................277
Using Code to Create Your Search String..................................................280
Calling the Google APIs ...............................................................................285
Displaying a Results Set ..............................................................................286
Creating the form................................................................................286
Calling the doGoogleSearch API .......................................................287
Displaying search results ..................................................................288
Saving search results .........................................................................291
Cycling through Multiple Results...............................................................292
Chapter 18: Understanding and Following Trends . . . . . . . . . . . . . . .295
Following the Spirit of the Times ...............................................................295
Creating a User Interface to Track Results ...............................................298
Returning the Estimated Hits for a Keyword............................................299
Getting Multiple Results..............................................................................301
Tracking Results over Time ........................................................................303
Displaying Results........................................................................................306
Part V: The Part of Tens .............................................309
Chapter 19: Ten (Mostly Really Useful)
Google Research Resources . . . . . . . . . . . . . . . . .311
CrossRef Search............................................................................................311
Enterprise Search Center ............................................................................312
Everything Google........................................................................................312
Google Zeitgeist............................................................................................313
Googlefight....................................................................................................313
Googlewhack.................................................................................................314
Open Directory Project Help Central ........................................................314
PageRank Explained.....................................................................................315
ResearchBuzz!...............................................................................................315
TouchGraph GoogleBrowser ......................................................................316
Chapter 20: Ten Tools That Use the Google APIs . . . . . . . . . . . . . . . . .317
Amoebacode.................................................................................................317
Anacubis........................................................................................................318
CapeMail........................................................................................................319
Flash Search with Google ............................................................................319
GARBO, GAPS, and GAWSH.........................................................................321
Google Alert ..................................................................................................321
Mangle ...........................................................................................................321
MetaLib..........................................................................................................322
Speegle...........................................................................................................322
XooMLe..........................................................................................................323
Part VI: Appendixes...................................................325
Appendix A: Using the SDK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .327
Downloading and Unzipping the SDK........................................................327
Understanding the SDK Contents ..............................................................328
Appendix B: Building a Research Application
with Visual Basic .NET . . . . .. . . . . . . . . . . . . . . . . . .331
Adding a Web Reference..............................................................................331
Changing the Name of the Proxy Class .....................................................332
Creating an Instance of the Class...............................................................334
Using the Web Methods (The APIs)...........................................................334
Searching Google................................................................................334
Retrieving a page from the cache.....................................................335
Getting a spelling suggestion ............................................................335
Displaying Results........................................................................................336
Showing search results......................................................................336
Showing the size of a page in the cache ..........................................338
Showing a spelling suggestion..........................................................338
Appendix C: Using the Google APIs with Java . . . . . . . . . . . . . . . . . .339
Getting Your Java Ducks in a Row..............................................................339
Understanding issues with Java .......................................................339
Java compilers and editors ...............................................................340
Running the Test ..........................................................................................342
Using the Google Library ............................................................................344
Building a Java application in NetBeans .........................................344
Creating the visual interface .............................................................345
Coding the application ......................................................................346
Moving on............................................................................................349
Index........................................................................351
Screenshot
Purchase Now !
Just with Paypal
Just with Paypal
Product details
Price
|
|
|---|---|
Pages
| 385 p |
File Size
|
11,165 KB |
File Type
|
PDF format |
ISBN-13
ISBN-10 | 978-0-7645-7809-0 0-7645-7809-X |
Copyright
| 2005 by Wiley Publishing, Inc |
●▬▬▬▬▬❂❂❂▬▬▬▬▬●
●▬▬❂❂▬▬●
●▬❂▬●
●❂●
═════● ●═════
