“ Spanner : Google’s Globally – Distributed Database “

Spanner is a NewSQL created by google. It is a distributed relational database that can distribute and store data in google’s big table storage system in multiple data centers. Spanner is Google’s scalable, multi-version, globally distributed, and synchronously-replicated database. It is the first system to distribute data at global scale and support externally-consistent distributed transactions. Spanner provides the scalability that enables you to store a few trillion database rows in millions of nodes distributed to hundreds of data centers. When you read data, spanner connects you to data center that is geographically closest and similarly when we write the data, it distributes and stores it to multiple data centers. If in case the data center we try to access has a failure we can read the data from another data center that has a replica of the data.

Replication is used for global availability and geographic locality clients automatically failover between replicas. Applications can use spanner for high availability even in the face of wide-area natural disasters, by replicating their data within or even across continents. Spanner’s main focus is managing cross-datacenter. Many applications at google have chosen to use Megastore because of its data model and support for synchronous replication despite poor throughput. Applications can specify con-straints to control which datacenters contain which data, how far data is from its users how far replicas are from each other an how many replicas are maintained can also be known.

Spanner has two features that are difficult to implement i.e providing externally consistent reads and writes, and globally-consistent reads across the database at a time-stamp. If a transaction t1 commits before another transaction t2 starts, then t1’s commit timestamp is smaller than t2’s. Spanner is the first system to provide this guarantees at global scale.A spanner deployment is called a universe. Spanner is organized as a set of zones, where each zone is the rough analog of a deployment of bigtable. Zones can be added to or removed from running system as new datacenters when brought into service and old ones are being turned off. As a globally-distributed database, Spanner provides several interesting features. First, the replication configurations for data can be dynamically controlled at a fine grain by applications. Applications can specify constraints to control which datacenters contain which data, how far data is from its users (to control read latency), how far replicas are from each other (to control write latency), and how many replicas are maintained (to control durability, availability, and read performance). Data can also be dynamically and transparently moved between datacenters by the system to balance resource usageacross datacenters.

These components are empowered by the way that Spanner allocates comprehensively important confer timestamps to exchanges, despite the fact that exchanges may be dispersed. The timestamps reflect serialization request. Likewise, the serialization request fulfills outer consistency (or proportionately, linearizability [20]): if an exchange T1 confers before another exchange T2 begins, then T1's confer timestamp is littler than T2's. Spanner is the to start with framework to give such ensures at worldwide scale. The key empowering influence of these properties is another TrueTime Programming interface and its usage. The API specifically uncovered clock vulnerability, and the sureties on Spanner's timestamps rely on upon the limits that the usage gives. On the off chance that the vulnerability is huge, Spanner backs off to endure that vulnerability. Google's bunch administration programming gives a usage of the TrueTime Programming interface. This usage keeps vulnerability little (for the most part under 10ms) by utilizing different cutting edge clock references (GPS and nuclear timekeepers).

The replication and distributed transactions have been layered onto big-table based implementation. This can viewed with the help of spanserver software stack, here in each spanserver is responsible for 100 to 1000 instance of a data structure called a tablet. Here spanner assigns timestamps to data since there are multiple versions of data. Spanner in order to support replication, each spanserver implements a single Paxos state machine on top of each tablet. The current spanner implementation logs every paxos write twice, basically first time in tablet’s log and later in the paxos log. The spanserver implements lock table in order to obtain the concurrency control.

A registry is likewise the littlest unit whose geographic replication properties (or arrangement, for short) can be indicated by an application. The outline of our arrangement determination dialect isolates obligations for overseeing replication setups. Overseers control two measurements: the number and sorts of imitations, and the geographic arrangement of those copies. They make a menu of named choices in these two measurements (e.g., North America, repeated 5 routes with 1 witness). An application controls how information is repeated, by labeling every database and/or singular registries with a mix of those choices. For instance, an application may store every end-client's information in its own index, which would empower client An's information to have three copies in Europe, and client B's information to have five imitations in North America.

Spanner implementation supports a bucketing abstraction called a directory, which is a set of contiguous keys that share a common prefix. Supporting these directories allows applications to to control the locality of their data. Directories movement is enabled even when clients operations are going on, and speed of the data movement could be fifty mega bytes in few seconds. Directory is also smallest unit whose properties could be specified by an application. Here in the directory the administrators take care of two dimensions : the number and types of replicas and the geographical placement of it.

Spanner exposes some set of data features to applications. Data model is layered on top of the directory-bucketed key-value mappings supported by implementation. Each database contains many number of tables. And tables consists of rows, columns, and values in it. Spanner’s datamodel contains row names and primary key is allotted to it. And as said previously rows contains some values for their existence. CREATE TABLE Users { uid INT64 NOT NULL, email STRING
} PRIMARY KEY (uid), DIRECTORY;

The above given querry is an example for spanner schema.

TT.now() method returns a TTinterval that is guaranteed to contain the absolute time. Similarly TT.after(t) returns true if t was definitely passed and finally TT.before(t) returns true if t has definiteky not arrived. Here The TT.after() and TT.before() methods are said to be convenience wrappers around TT.now(). The time references used by the True Time are atomic clocks. Atomic clocks can fail over long periods of time can drift significantly due to frequency error. True Time is implemented by a set of time master machines.The majority of the masters posses GPS receivers with antennas. The remaining masters are equipped with atomic clocks.

LastTS() as characterized above has a comparable shortcoming: if an exchange has quite recently dedicated, a non-clashing readonly exchange must at present be doled out sread in order to take after that exchange. Therefore, the execution of the read could be postponed. This shortcoming can be cured comparably by increasing LastTS() with a fine-grained mapping from key reaches to submit timestamps in the lock table. (We have not yet actualized this improvement.) At the point when a read-just exchange arrives, its timestamp can be doled out by taking the most extreme estimation of LastTS() for the key reaches with which the exchange clashes, unless there is a clashing arranged exchange.

The spanner implementation supports read-write transactions and read-only transactions. Standalone write are implemented as read-write transactions. Non-standalone reads are implemented as read-only transaction. For both read-only transactions and snapshots reads, commit is inevitable since time stamp was chosen, unless data at that timestamp was collected. When any server fails, clients can continue their process of querry internally on different server by choosing or repeating the timestamp and also the current read position. Transactional reads and writes use two-phase locking. When all locks have acquired prior release of any lock, for given transaction, spanner assigns timestamp to paxos that represents transaction commit. A single reader assigns timestamp monotonically in increasing order. And whenever a timestamp is assigned Smax is advanced to S.here each paxos state machine has a safe time Ttmsafe and Tpaxossafe is simpler, writes will no longer occur at or below with respect to paxos. Tsafe = min (Tpaxossafe , Ttmsafe).

tPaxos sheltered as characterized above has a shortcoming in that it can't advance without Paxos composes. That is, a preview perused at t can't execute at Paxos bunches whose last compose happened before t. Spanner addresses this issue by exploiting the disjointness of pioneer lease interims. Each Paxos pioneer propels tPaxos safe by keeping an edge above which future composes' timestamps will happen: it keeps up a mapping MinNextTS(n) from Paxos succession number n to the base timestamp that may be relegated to Paxos grouping number n + 1. A copy can progress tPaxos

In read-write transaction, mwrites that occur in a transactions that comes under the client, as a result reads in transaction don’t see the effects of write in the transactions. Reads within read-write are bound to avoid deadlocks. When client has completed all reads and buffered all writes, it begins two-phases commit message to each participant;s leader writes. Spanner requires a scope expression for every read-only transaction, which is an expression that summarizes key thet will read entire transaction. A spanner schema-change transaction is generally non-blocking variant of a standard transaction. As a result, schema changes across thousands of servers with minimal disruption to other concurrent activity. With true time, defining schema change to happen at t would be meaningless. Information can be stored in the lock table, which already maps key ranges to be clocked against fine-grained safe time for key ranges with read conflicts. The execution of the read could be delayed. When a read-only transaction arrives, timestamp can be assigned taking maximum value of LastTS() for the key ranges unless there is a conflicting prepared transaction.

The spanner’s Performance is measures with respect to replication, transactions and availability. And also we present some benchmarks for spanner. Where some measurements were taken on machines, each spanserver was run on 4GB ram and cores AMD 2200MGHZ. The information demonstrates that these two components in deciding the base estimation of are for the most part not an issue. Be that as it may, there can be huge tail-dormancy issues that cause higher estimations of . The lessening in tail latencies starting on March 30 were because of systems administration changes that lessened transient system join blockage. The increment in on April 13, around one hour in term, come about because of the shutdown of 2 time experts at a datacenter for routine upkeep. We keep on examining also, evacuate reasons for TrueTime spikes.

Clients and zones were placed in a set of datacenters with network distance of less than 1ms.Operations were stand alone reads and writes of 4KB. For latency operations and throughput operations on cpu’s. Single-read read-only transaction only execute at leaders because timestamp assignment must happen at leaders. Read-only transaction throughput increase with number of replicas, were randomly distributed among the zones. Two questions were answered with respect to True-time. Our machine statistics show that bad cpu’s are 6 times more likey than bad clocks. Those are lock issues, relative to much more serious problems. Spanner started being experimentally evaluated during 2011, under google’s advertising backend called F1. This was originally based on MYSQL databse that was manually shared many ways. The last resharding to this spanner took over 2 years of time. The F1 team chose to use spanner for several reasons. There were several steps which need to carried out First, spanner removes the need to manually re-shared, second, spanner provides replication, failover was difficult. The F1 team also needed secondary indexes on their data and was bale to implement their own consistant global ideas using spanner related transactions.

In conclusion, Spanner uses ideas from two communities database community and semi-relational interface transaction, an SQL- based query language from systems was used. Spanner shouldalso focus on databse features that bigtable was missing. And an another aspect of the spanner design stands out is that it is possible to build the distributed systems with much more stronger semantics and when tighter bounds stronger semantics decreases. As a community we should no longer depend on loosely synchronized clocks and weak api’s in designing distributed algorithms.

There has constant work going on based on this spanner with F1 team to transition google’s advertising backend from MYSQL to spanner, now there is current process trying to improve functionality and performance of backup. Given that we expect many applications to replicate datacenters Truetime may notice performance Timemaster querry intervals can be reduced and better clock crystals or possibly even avoided through alternate time-distribution technology.Finally there were some areas for improvement although spanner is scalable in number of nodes algorithms and datastructures from database could improve single node and also moving data automatically between datacenters in response to changes in client load but to make effective, we would also need to coordinated fashion.