Scalability and reliability in a distributed search engine

Nobuyoshi Sato; Minoru Udagawa; Minoru Uehara; Yoshifumi Sakai; Hideki Mori

doi:10.1109/ICPADS.2002.1183378

Scalability and reliability in a distributed search engine

Nobuyoshi Sato, Minoru Udagawa, Minoru Uehara, Yoshifumi Sakai, Hideki Mori

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

We have developed a distributed search engine, called cooperative search engine (CSE), in order to retrieve fresh information. In CSE, a local search engine located in each Web server makes an index of local pages. A meta search server integrates these local search engines in order to realize a global search engine. In such a way, the communication delay occurs at retrieval time. So, it is thought to be difficult to search quickly. However we have developed several speedup techniques in order to realize real time retrieval. In addition, the meta server is a single point of failure in CSE. So, we propose redundancy of meta search servers in order to increase availability of CSE. In this paper we describe scalability and reliability of CSE and their evaluations.

Original language	English
Title of host publication	Proceedings - 9th International Conference on Parallel and Distributed Systems, ICPADS 2002
Publisher	IEEE Computer Society
Pages	57-62
Number of pages	6
ISBN (Electronic)	0769517609
DOIs	https://doi.org/10.1109/ICPADS.2002.1183378
Publication status	Published - 2002 Jan 1
Externally published	Yes
Event	9th International Conference on Parallel and Distributed Systems, ICPADS 2002 - Taiwan, China Duration: 2002 Dec 17 → 2002 Dec 20

Publication series

Name	Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS
Volume	2002-January
ISSN (Print)	1521-9097

Other

Other	9th International Conference on Parallel and Distributed Systems, ICPADS 2002
Country/Territory	China
City	Taiwan
Period	02/12/17 → 02/12/20

Keywords

Delay effects
Distributed computing
Information retrieval
Metasearch
Network servers
Redundancy
Scalability
Search engines
Web pages
Web server

ASJC Scopus subject areas

Hardware and Architecture

Access to Document

10.1109/ICPADS.2002.1183378

Cite this

Sato, N., Udagawa, M., Uehara, M., Sakai, Y., & Mori, H. (2002). Scalability and reliability in a distributed search engine. In Proceedings - 9th International Conference on Parallel and Distributed Systems, ICPADS 2002 (pp. 57-62). [1183378] (Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS; Vol. 2002-January). IEEE Computer Society. https://doi.org/10.1109/ICPADS.2002.1183378

Scalability and reliability in a distributed search engine. / Sato, Nobuyoshi; Udagawa, Minoru; Uehara, Minoru; Sakai, Yoshifumi; Mori, Hideki.

Proceedings - 9th International Conference on Parallel and Distributed Systems, ICPADS 2002. IEEE Computer Society, 2002. p. 57-62 1183378 (Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS; Vol. 2002-January).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Sato, N, Udagawa, M, Uehara, M, Sakai, Y & Mori, H 2002, Scalability and reliability in a distributed search engine. in Proceedings - 9th International Conference on Parallel and Distributed Systems, ICPADS 2002., 1183378, Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS, vol. 2002-January, IEEE Computer Society, pp. 57-62, 9th International Conference on Parallel and Distributed Systems, ICPADS 2002, Taiwan, China, 02/12/17. https://doi.org/10.1109/ICPADS.2002.1183378

Sato N, Udagawa M, Uehara M, Sakai Y, Mori H. Scalability and reliability in a distributed search engine. In Proceedings - 9th International Conference on Parallel and Distributed Systems, ICPADS 2002. IEEE Computer Society. 2002. p. 57-62. 1183378. (Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS). https://doi.org/10.1109/ICPADS.2002.1183378

Sato, Nobuyoshi ; Udagawa, Minoru ; Uehara, Minoru ; Sakai, Yoshifumi ; Mori, Hideki. / Scalability and reliability in a distributed search engine. Proceedings - 9th International Conference on Parallel and Distributed Systems, ICPADS 2002. IEEE Computer Society, 2002. pp. 57-62 (Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS).

@inproceedings{02e4e130a2614c3c87166784b535e537,

title = "Scalability and reliability in a distributed search engine",

abstract = "We have developed a distributed search engine, called cooperative search engine (CSE), in order to retrieve fresh information. In CSE, a local search engine located in each Web server makes an index of local pages. A meta search server integrates these local search engines in order to realize a global search engine. In such a way, the communication delay occurs at retrieval time. So, it is thought to be difficult to search quickly. However we have developed several speedup techniques in order to realize real time retrieval. In addition, the meta server is a single point of failure in CSE. So, we propose redundancy of meta search servers in order to increase availability of CSE. In this paper we describe scalability and reliability of CSE and their evaluations.",

keywords = "Delay effects, Distributed computing, Information retrieval, Metasearch, Network servers, Redundancy, Scalability, Search engines, Web pages, Web server",

author = "Nobuyoshi Sato and Minoru Udagawa and Minoru Uehara and Yoshifumi Sakai and Hideki Mori",

year = "2002",

month = jan,

day = "1",

doi = "10.1109/ICPADS.2002.1183378",

language = "English",

series = "Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS",

publisher = "IEEE Computer Society",

pages = "57--62",

booktitle = "Proceedings - 9th International Conference on Parallel and Distributed Systems, ICPADS 2002",

note = "9th International Conference on Parallel and Distributed Systems, ICPADS 2002 ; Conference date: 17-12-2002 Through 20-12-2002",

}

TY - GEN

T1 - Scalability and reliability in a distributed search engine

AU - Sato, Nobuyoshi

AU - Udagawa, Minoru

AU - Uehara, Minoru

AU - Sakai, Yoshifumi

AU - Mori, Hideki

PY - 2002/1/1

Y1 - 2002/1/1

N2 - We have developed a distributed search engine, called cooperative search engine (CSE), in order to retrieve fresh information. In CSE, a local search engine located in each Web server makes an index of local pages. A meta search server integrates these local search engines in order to realize a global search engine. In such a way, the communication delay occurs at retrieval time. So, it is thought to be difficult to search quickly. However we have developed several speedup techniques in order to realize real time retrieval. In addition, the meta server is a single point of failure in CSE. So, we propose redundancy of meta search servers in order to increase availability of CSE. In this paper we describe scalability and reliability of CSE and their evaluations.

AB - We have developed a distributed search engine, called cooperative search engine (CSE), in order to retrieve fresh information. In CSE, a local search engine located in each Web server makes an index of local pages. A meta search server integrates these local search engines in order to realize a global search engine. In such a way, the communication delay occurs at retrieval time. So, it is thought to be difficult to search quickly. However we have developed several speedup techniques in order to realize real time retrieval. In addition, the meta server is a single point of failure in CSE. So, we propose redundancy of meta search servers in order to increase availability of CSE. In this paper we describe scalability and reliability of CSE and their evaluations.

KW - Delay effects

KW - Distributed computing

KW - Information retrieval

KW - Metasearch

KW - Network servers

KW - Redundancy

KW - Scalability

KW - Search engines

KW - Web pages

KW - Web server

UR - http://www.scopus.com/inward/record.url?scp=84948776944&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84948776944&partnerID=8YFLogxK

U2 - 10.1109/ICPADS.2002.1183378

DO - 10.1109/ICPADS.2002.1183378

M3 - Conference contribution

AN - SCOPUS:84948776944

T3 - Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS

SP - 57

EP - 62

BT - Proceedings - 9th International Conference on Parallel and Distributed Systems, ICPADS 2002

PB - IEEE Computer Society

T2 - 9th International Conference on Parallel and Distributed Systems, ICPADS 2002

Y2 - 17 December 2002 through 20 December 2002

ER -

Scalability and reliability in a distributed search engine

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this