THE SQL Server Blog Spot on the Web

Welcome to - The SQL Server blog spot on the web Sign in | |
in Search

Joe Chang

Low Latency Memory

For the last twenty years, the standard practice has been to scale server performance with multi-processor (MP) systems. What is not widely understood is that after processors integrated the memory controller, the initial step from single processor to 2-way has only moderate scaling due to the introduction of non-uniform memory access (NUMA) in the MP system architecture. Scaling versus cores within a single processor however, can be excellent. The new practice should now be to employ single processor systems for the very large majority of situations, with multi-processor systems relegated to extreme boundary cases.

Once we accept that the single processor system is the correct strategy, the next conclusion is that low latency DRAM should be pursued. Over the last twenty years, the data transfer rate for memory has increased from 100-133MHz in the SDRAM generation to over 2666MT/s in DDR4. But the full access latency (or row cycle time) has barely changed from perhaps 60ns to 45ns. This not because some fundamental limit of DRAM has been reached. Rather it is a deliberate decision to prioritize manufacturing cost. It was presumed that low latency would not have significant incremental market value to justify the higher cost.

A significant portion of the memory latency in multi-processor servers occurs in the system elements outside of DRAM. A reduction in latency on the DRAM chip may not impact total memory latency in the MP system sufficiently to outweigh the cost. However, in a single processor system, the impact is sufficient to justify even a very large increase in DRAM cost. The first opportunity is for low latency DRAM that would be compatible with the existing memory interface of current or near-term next generation processors, be it DDR4 now or DDR5 in a couple of years.

The next step is for memory with a new optimized interface, which must be implemented in conjunction with the processor. The most obvious change is to de-multiplex the address bus, basically RL-DRAM, but optimized for server systems. Memory latency is so important that it is likely even SRAM cost structure is viable, but the discussion here focuses on DRAM.

Memory Latency in Single and Multi-Processor Systems

There are three die options in the Skylake based Intel Xeon SP: LCC, HCC and XCC. The figure below shows representation of the Xeon SP XCC model. Memory access latency on L2 miss consists of L3 plus the DRAM access. L3 latency may be less than 19ns on the LCC and HCC die.


Intel cites 19.5ns for the XCC die. 7-cpu reports memory access for Skylake X as L3 + 50ns = 69ns for DDR4-3400 16-18-18-36. On more conservative ECC memory at 2666MT/s 18-18-18 timing, the full memory access might be 76ns, or perhaps slightly higher.

The figure below, from Anandtech, source from an Intel slide?, shows memory latency for a 2-way system with Xeon SP 8180 28-core processors. Also see Sizing Up Servers....


Note that local node memory latency is shown as 89ns, significantly higher than latency on the single processor system. Remote node memory latency is 139ns. I am assuming that this difference is due to remote node cache coherency, but processor architects and engineers are welcome to elaborate on this. There is a slide in Notes on NUMA Architecture, Intel Software Conference 2014 Brazil stating that in local memory access, the memory request is sent concurrently with the remote node snoop, implying minimal penalty, and yet there is a difference of 13ns?

Be aware that different values are cited by various sources. The figure below is from


Even if a database or other application had been architected to achieve a high degree of memory locality on a NUMA system, there is still a memory latency penalty on the multi-processor system relative to the single processor system.

Almost zero real-world databases have been architected to achieve memory locality on a NUMA system. I have heard that a handful of Oracle environments went to effort to re-architect for RAC, which is the same type of architecture necessary to achieve memory locality on NUMA. Most environments should see average memory latency of 114ns on a 2-way Xeon SP system based on 50/50 local-remote node mix. Average memory latency on a 2-way Xeon E5 v4 system might be 120ns.

Memory Latency and Performance

The main line Intel processor cores manufactured on the original 14nm process can operate with base frequency at the high 3GHz level. Cores manufactured on the 14nm+ process can operate at the low 4GHz level, base. In the high core Xeon models, this is typically de-rated to 2-2.5GHz base frequency with turbo boost at about 3GHz.

At 2.5GHz, the CPU clock is 0.4ns. Single processor memory latency of 76ns corresponds to 190 CPU-cycles. Local node memory access in a multi-processor system of 89ns is 222.5 cycles and remote node latency of 139ns is 347.5 cycles. Average 2-way memory access based on 50/50 local-remote node split is 114ns or 285 cycles.

If even a very smaller percentage of code involves a round-trip memory access to determine the next operation, then most CPU cycles are spent as no-ops. In this case, performance is essentially the inverse of memory latency. A 30% reduction in memory latency corresponds to 1.4X performance and 50% to 2X. See Memory Latency for more on this topic.


The Dual In-line Memory Module has been a standard for many years now. In the figure below, a memory module is shown with 9 DRAM chip packages forming a 72-bit wide word or line.


Of this, 64-bits are data and the 8 extra bits are for ECC. There is an extra chip for registered DIMMs. Desktop system used unbuffered DIMMs and do not have ECC.

A module can be double-sided. When there are chip packages for two separate sets, then the DIMM is said to be having 2 ranks. In case above, there is one set on each side of the DIMM, hence dual.

The figure below shows two separate sets on each side, for a total of four sets.


In this case, the DIMM is rank 4. The DIMM datasheet will have the actual configuration as there are exceptions. Below is a DIMM probably of rank 4.


Each package can have stacked DRAM die. The image below is probably NAND die as it appears to be stacked 16 high. DRAM packages with stacked die are usually 2 or 4 high?



DRAM - preliminary

The DRAM chip is an enormously complicated entity for which only some aspects are briefly summarized here. See Onur Mutlu and other sources for more.

Below is a simplified rendering of the Micron 8Gb DDR4 in the 2G x 4 organization. See the Micron datasheet for a more complete and fully connected layout (DDR4 8Gb 2Gx4).


The die is sub-divided into 4 bank groups. Each group has 4 banks for a total of 16 banks. Within a bank, there are rows and columns. The row is a word-line and the column is a bit-line.

The 2G x 4 organization means there are 2G words of 4-bits. The address bus has 21 signals. Two bits are for addressing the bank groups, and two bit are for the banks. The remaining 17 signals are multiplexed for the row and column addresses. All 17 bits are used for the row (128x1024 = 131,072) address. Only 10 bits are used for the column address. The total address is then 4 + 17 + 10 = 31 bits, sufficient for 2G words.

Notice that only 7 bits of the column address go into the column decoder. Eight words are fetched from the bank on each access

The diagrams below are from Bounding Worst-Case DRAM Performance on Multicore Processors, on Oak Central. The original source is: "Memory Systems: Cache, DRAM, Disk " by Bruce Jacob et. al. (2008). The first image shows "Command and data movement for an individual DRAM device access on a generic DRAM device."


The next figure is "A cycle of a DRAM device access to read data."


The three DDR timing parameters commonly cited are CL, tRCD, and tRP. A fourth number that may be cited after the first three is tRAS. Wikipedia Memory timings has definitions for these terms. "The time to read the first bit of memory from a DRAM with the wrong row open is tRP + tRCD + CL." The tRCD, CL and tRP element are often identical and in the 13-14ns range. In DDR4, the data burst phase transfers 8 words. For DDR4-2666, the command-address clock is 0.75ns and the data transfer rate is one word every 0.375ns, so 8 transfers takes 3ns.

The data restore phase occurs shortly tCAS (CL) and overlaps with tBurst. There is a gap between the column read and the (array) precharge periods. As mentioned earlier, DRAM is very complicated. For database transaction processing and other pointer-chasing-code, the parameter of interest is tRC. This value is sometimes not cited, and it cannot be determined by the three commonly cited timing parameters.

Micron SDRAM through DDR4

The table below shows some organization details for selected Micron DRAM products from SDRAM through DDR4. The year column is the year of the datasheet, which might be a revision published after initial product launch.

Rows x
SDRAM256M64Mx4413118K x 2K199913315ns60ns
DDR512M128Mx4413128K x 4K200040015ns55ns
DDR22G512Mx48151132K x 2K2006106613.13ns54ns
DDR31G256Mx48141116K x 2K2008186613.91ns48ns
DDR34G1Gx48161164K x 2K2017213313.09ns47ns
DDR48G2Gx4161710128K x 1K2017320013.75ns46ns

Most of the parameters above are in the DRAM chip datasheet, but tRC is usually found in the DIMM datasheet, and sometimes buried in the back pages. The timing column is the CL-tRCD-tRP amalgamated into a single value and converted from (command/address) clock cycles to nanoseconds.

The CL-tRCD-tRP values decreased only slightly from SDRAM to DDR2. From DDR2 to current DDR4, they have not changed for the registered ECC DIMMs used in server systems. There are unbuffered DIMMs made with specially screened DRAM without ECC having lower (better) values. These products also have heat sinks for improved cooling. Presumably, it would not be feasible to rely on this approach for server systems.

The 256Mb SDRAM datasheet is listed as 1999. This could be a 250 or 180nm manufacturing process. The 8Gb DDR4 is probably a 2016 or 2017 product, possibly on a 22nm or 2x process. From 256Mb to 8Gb, there are 5 doublings of density. In between 180nm and 22nm, are five manufacturing processes: 130, 90, 65, 45 and 32nm.

The manufacturing process used to mean the transistor gate length. However, in the last 15 or so years, it is just an artificial placeholder representing a doubling of transistor density between generations. In the Intel 22 to 14nm process transition, the density increased by 2.7X (Intel's 10nm).

It would seem that one doubling of DRAM density between 256Mb at 180nm and 8Gb at 22nm is missing, but this could be the upcoming 16Gb single die. There is a convention in which the DRAM bit cell size is specified as xF2 and F is the process linear dimension.

DRAM Die Images

In Under the Hood: The race is on for 50-nm DRAM, the Samsung 1Gb DDR2 is mentioned as 50nm class in 2009 as a 6F2-based cell design, die image shown below. In this case, 50nm class is 58nm.


Also is Under the Hood, "On the other hand, Hynix's 8F2 cell design showed a 16.5 percent larger cell than Samsung's. It should be noted, that despite the larger cell size, Hynix's 1-Gbit DDR2 SDRAM achieved an impressive chip size of 45.1 mm2, only 2.7 percent larger than Samsung's 1-Gbit DDR2 SDRAM."

Semiconductor Manufacturing & Design has a die image of the Samsung 1Gb DDR3 in How to Get 5 Gbps Out of a Samsung Graphics DRAM, shown below.


Embedded has a discussion on Micron SDRAM and DDR in Analysis: Micron cell innovation changes DRAM economics.

DRAM Latency

Two items are mentioned as having significant impact on DRAM latency. One is the bank size. Another is the multiplexing of row and column addresses. In principle, it is a simple matter to sub-divide the DRAM chip into more banks, as the design of each bank and its control logic is the same. This would however increase the die size as there would be more of the control logic associated with each bank. Earlier we said that DRAM manufacturers understood that the market prioritized cost over latency.

In SDRAM 256Mbit, the bank size is 8Kx2K = 16M words, with 4 banks at x4 word. (I am using the term word to mean the data path out. The could apply to a DRAM chip at 4, 8 or 16-bits, and a DIMM at 64 or 72-bits.) For DDR4 8Gbit, bank size is 128Kx1K = 128M with 16 banks at x4 word size.

There is some improvement in latency between the manufacturing process for SDRAM in 1999 and DDR4 in 2017. Long ago, the expectation was a 30% reduction in transistor switching time corresponding to 40% increase in frequency for a logic. From about the 90nm process forwards, transistor performance improved at a much diminished rate.

In DRAM, there is both the logic sub-units made of transistors and the bit cell array made of transistors and capacitors. I do not recall mention of capacitor performance versus process generation. Presumably it is minimal? Still, this allowed the main timing parameters to remain at 13-14ns each as bank size increased from 16M to 128M at 4-bit data width.

A single bank can only transfer one 8-word burst every tRC. This takes 4 clocks, as DDR transfers 2 data words every clock. The tRC is somewhat longer than tRCD + CL + tRP. For DDR4-2666, these components are 19 clocks each for a total of 57 clocks at 0.75ns per clock. The cited tRC at 3200 MT/s is 45.75 and at 2933 MT/s is 46.32ns. Presumably then tRC at 2666 MT/s is either 61 clocks for 45.75ns or 62 clocks for 46.5ns. (61 clocks for tRC is 57 + 4?)

A single bank can transfer data for only 4 out of 61 or 62 clock cycles. DDR4 currently has 16 banks. So, it would seem that sustained memory bandwidth was important and warranted an increase in the number of banks. But latency was not deemed sufficiently important to further increase the number of banks.

Reduced Latency DRAM

Micron has a RLDRAM 3 product at 1.125Gb. One organization is 4M x18 and 16 banks. The datasheet is 2015 so this might be on the same manufacturing process as the 2Gb DDR3. The RLDRAM package is 13.5mm x 13.5mm versus 8mm x 14mm for the 2Gb in x16 form. This might be due to the difference in pins, 168 versus 96. The two products might have the same die size or the 1.15Gb RLDRAM could have a larger die size than the 2Gb DDR3, and may or may not be on a common manufacturing process.

The 1.125Gb RL-DRAM has 16 banks. Each bank is 16K rows by 256 columns = 4M words, or 72M bits.

The appropriate comparison for this is the 2Gbit DDR3 in 128M x 16 organization. Then the comparable DDR3 chip is 8 banks. Each bank is 16K x 1K = 16M words at 16 bit word = 256M bits. So, roughly, the bank size is 3.55 times smaller in terms of bits per bank.

The second aspect of RLDRAM is that the row and column addresses are not multiplexed. For the 64M x 18 organization, there are 25 address bits, even though 26 bits are required to address 64M words. There are 4-bits for the bank address. The row address is 14-bits. The column address is 7-bits, but only 5-bits go into the Column decoder. The bank is shown as 16K x 32 x 8 (x18 data). The 5-bit going into the decoder corresponds to 32 columns. I am presuming that the address has 2 word granularity?

Between the smaller bank size and the non-multiplexed address, Micron cites their RL-DRAM as having tRC minimum value of 6.67ns (8-cycles of 2400T/s, or 0.833ns per data transfer). But why only cite the minimum value? We are mostly interested in average value and possible the min-max range, excluding refresh, which all DRAM must have.

My guess would be that by having extra banks, the tRP period can be hidden if access are randomly distributed between banks. If so, then the smaller banks reduces the 13-14ns timings to 6.67ns?

It is presumed that both the smaller bank size and non-multiplexed address contributes to significantly lower latency. Some details on how much each aspect contributes would be helpful.

As we are accustomed to multi-GHz processors, it might seem strange that the multiplexed row and column address has much of a penalty. The tRCD and CAS latency are each 13-14ns in mainstream DRAM. In this regard, we should recall that processors have been at 10 pipeline stages since Pentium Pro in 1995. The post-Pentium 4 Intel processors are probably 14-16 pipeline stages, though Intel no longer shows the number of stages in their micro-architecture diagrams.

In this regard, the full start to finish time to execute an instruction is on the order of 5ns. Then factor in that DRAM has a very different manufacturing process than logic, which is not optimized for performance. It is presumed that the logic on DRAM is not pipelined except for the data transfer sub-units? (the DRR logic core runs at 1/8th of the data transfer rate, or 1/4 of the command/address clock. On DDR-2666, the core runs at 333MHz.)

Low Latency DRAM

As mentioned earlier, we are interested in two separate approaches to low latency DRAM for server systems in handling transaction processing type workloads. The near-term approach is to retain the existing mainstream DDR4 interface and DDR5 when appropriate. Ideally, the new special memory could be compatible with now or then existing processors designed for conventional memory. But some divergence may be necessary and is acceptable.

This would preclude a non-multiplexed address bus. The number of banks would be increased from 16 in the 8Gb DDR4 presumably to 64 or more. The memory controller would have to know that there are more bits for bank addressing, which is why this new memory may or may not work (at full capability?) in existing processors. (The memory controller tries to schedule requests so that accesses do not go to a given bank within a tRC interval.)

But it is possible the next generation processor could work with both conventional and the new high bank count DRAM memory. (Before conventional memory was made with multi-banks, there was a Multibank DRAM product from Mosys.)

The low latency DRAM die would be larger than a similar density conventional DRAM chip. Either the new memory would have lower DIMM capacity options or the DIMM form factor would have to allow for a taller module. A 2X reduction is capacity would not be a serious issue. If the difference in areal size were 4X, then the larger form factor would be preferable.

In the longer term strategy, more avenues for latency reduction are desired. The RL-DRAM approach of non-multiplexed row and column addresses is the right choice. Somewhere its mentioned that the RL-DRAM interface is not very different from SRAM. This could be an additional option that I looked at in SRAM as Main Memory.


We need to wake up to the fact that scaling performance with multi-processor systems should no longer be the standard default approach. Most situations would benefit greatly from single processor system. On a hardware cost only evaluation, the cost-performance value could go either way. Two 20 core processors cost less than one 28-core, but one 24-core is about the same as two 16-core processors. When software per-core licensing is factored, the single processor system advantage is huge. Once we accept the single processor system approach, it is easy to then realize that a new low latency memory provides more huge additional value.

This needs to be communicated to both DRAM and processor manufacturers. A cost multiplier of 2 or even 4X is well worth it if system level memory latency could be reduced by 20ns. A near-term future product with a backward compatibility option would mitigate risk. The longer term approach is a clean break from the past to do what is right for the future.


There is manufacturing variation in DRAM latency. Most mainstream DRAM is set at a conservative rating for high yield. Micron does have parts with slightly better latency, usually denoted with a E in the speed grade. Some companies (ex. G.Skill) offers memory made with specially screened parts. One product is DD4-4266 (2133 clock) at 17-18-18 timings. CL 17 at 2133MHz is 7.97ns. RCD and RP at 18 is 8.44ns. This strategy works for extreme gaming, allowing a few deep-pocket players to get an advantage. For servers, it is probably worth while to offer two or three binned parts. If two, then perhaps the top one-third as the premium and the rest as ordinary.

The other factor that affects latency is temperature. Lower temperature allows the charge state of the capacitor to be measured more quickly. Many DRAM parts lists 64ms, 8192-cycle refresh for up to 85C, and 32ms refresh for >85C to 95C. Onur's slides also says that lower temperature allows lower timing values. This is one reason why the special DIMMs for gaming have a heat sink. So, a question is whether we want this or even more elaborate cooling for our low latency strategy. Perhaps cooling should be used for the specially binned parts, and just normal cooling for the bulk parts.

Published Thursday, March 8, 2018 11:48 AM by jchang

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS



Steve Rogger said:

I am also facing this low latency issue. I could not find the solution so I am in great trouble now. Please help me to find the solution to this problem. Contact in

<a href="">iCloud support number</a> for more information.

July 19, 2018 5:30 PM

Peter said:

Within this rapid growth and development of the data age, we are able to notice the significance of software development and knowledge technology. Lots of people devote themselves to the introduction of software and application. At you can get help in promotion of your app. And increasingly more students are curious about mobile application industry. Apple launches Application Development Curriculum for school.

July 30, 2018 12:41 AM

Acacia said:

I appreciate your article, it is very helpful with my work.

August 5, 2018 9:15 PM

Asmund Nadja said:

August 14, 2018 3:44 AM

crazymichael said:

August 15, 2018 9:21 AM said:"> alexander mcqueen woman, alexander mcqueen,"> moncler kids jackets,"> adidas,"> golden goose v star, valentino,"> golden goose superstar,"> supreme package deals,"> valentino sandals,"> golden goose may,"> golden goose slide,"> supreme package deals,"> golden goose francy, golden goose,"> golden goose may, golden goose slide,"> nike x fragment men,"> nike air max 1 women, golden goose,"> golden goose may,"> nike air zoom men,"> nike air max tn men,"> alexander mcqueen sneakers,"> valentino pumps,"> golden goose superstar,"> golden goose francy,"> supreme crewneck,"> nike air zoom women,"> adidas pure boost dpr,"> golden goose slide,"> supreme shirt,"> adidas yeezy boost 750,"> nike air max 90 sneakerboot men,"> golden goose mid star,"> off white x vans,"> adidas eqt support,"> golden goose 2.12,"> air jordan cdp, golden goose mid star,"> nike air max 87 women, fashion shoes,"> air jordan 1,"> golden goose starter,"> valentino slingbacks,"> adidas yeezy boost 700,"> supreme pant,"> nike air force 1 women,"> nike air max 90 women,"> balenciaga women sneakers, golden goose, golden goose superstar, golden goose ball star,"> air jordan 2,"> jimmy choo heels,"> moncler scarves hats,"> golden goose francy, golden goose starter,"> golden goose starter,"> nike,"> air jordan 9,"> valentino sneakers, valentino sandals,"> supreme hoodies,"> supreme hoodie,"> jimmy choo flats, golden goose superstar, golden goose,"> golden goose francy, valentino pumps,"> nike sb men,"> valentino sandals,"> valentino slingbacks,"> supreme jackets, golden goose,"> nike flyknit air max women,"> golden goose may,"> valentino slides, valentino,"> nike air max dynasty women,"> common projects woman,"> adidas yeezy boost 750,"> brazil world cup jerseys,"> golden goose v star,"> nike free 3.0,"> adidas nmd human race,"> golden goose mid star, golden goose slide,"> moncler women coats,"> air jordan 8, valentino ballerinas,"> supreme jacket,"> adidas crazy explosive,"> moncler kids coats, golden goose,"> nike zoom speed trainer 3,"> valentino flats,"> golden goose 2.12,"> nike air max 2017 men,"> valentino sneakers,"> england world cup jerseys,"> supreme pants,"> common projects man,"> off white shoes,"> adidas iniki runner boost,"> portugal world cup jerseys, golden goose may, golden goose slide, golden goose starter, golden goose mid star,"> moncler women jackets,"> valentino ballerina flats,"> nike air max women,"> nike air huarache men,"> off white x nike,"> air jordan spizike,"> nike air max 1 men,"> air jordan dmp,"> supreme bag, yeezy boost 350 v 2,"> valentino ballerinas,"> golden goose slide,"> golden goose v star,"> supreme t shirts,"> nike free run men,"> golden goose francy,"> giuseppe zanotti sneakers,"> balenciaga shoes, valentino outlet,"> air jordan,"> nike air max 95 women,"> nike air max men,"> golden goose ball star, golden goose v star, valentino flats, valentino pumps,"> golden goose francy,"> golden goose slide,"> adidas nmd shoes,"> supreme crewnecks,"> nike sb women,"> off white accessories, golden goose, giuseppe zanotti,"> nike free 5.0, supreme outlet, golden goose starter,"> balenciaga city bags, valentino slingbacks,"> air jordan 3, supreme long tshirts, world cup jersey, golden goose slide, golden goose may,"> balenciaga men sneakers, golden goose mid star, golden goose, golden goose superstar, golden goose francy,"> nike roshe run men, valentino slides, valentino,"> air jordan 13, argentina world cup jerseys, valentino shoes, jimmy choo,"> valentino slides,"> balenciaga crossbody bags,"> air jordan 11, golden goose francy, balenciaga,"> nike air max 90 men, balenciaga tote bags, golden goose ball star, moncler outlet,"> golden goose starter, valentino sandals, valentino sneakers, golden goose starter, golden goose tennis, balenciaga shoulder bags, golden goose,"> air jordan 12,"> nike x fragment women, golden goose v star, golden goose francy, adidas yeezy boost, valentino pumps, off white,"> nike air max 87 men, balenciaga clutch bags, golden goose ball star,"> air jordan 7,"> golden goose superstar, valentino slides,"> nike lunar men, valentino slides,"> adidas ultra boost, moncler men coats, adidas yeezy boost 350 v2,"> nike roshe run women,"> air jordan 4,"> nike basketball, golden goose starter, golden goose v star,"> valentino pumps, supreme tshirt,"> adidas y3 pure boost, alexander mcqueen man, balenciaga, golden goose ball star, valentino sandals, france world cup jerseys, yeezy boost 750, germany world cup jerseys,"> air jordan womens, common projects, golden goose starter, valentino ballerinas, moncler women vests,"> nike vapormax women,"> air jordan 6 rings,"> air jordan 5, adidas yeezy boost 350, off white x adidas, valentino flats,"> nike air max dynasty men, valentino flats, valentino boots, valentino ballerinas,"> nike air max excellerate men,"> nike air max 2017 women,"> nike air force 1 men,"> adidas dame 4,"> nike free 4.0, valentino flats, yeezy boost infant, golden goose super star, golden goose 2.12, spain world cup jerseys, adidas yeezy boost infant, golden goose mid star,"> nike air max thea print women, valentino sandals, moncler men jackets, moncler men vests,"> nike air max thea print men, supreme shoes,"> golden goose tennis, golden goose v star,"> adidas harden vol 2, golden goose superstar,"> golden goose ball star, golden goose man,"> nike vapormax men, supreme scarf,"> adidas yeezy boost 350, golden goose mid star, golden goose v star, golden goose,"> nike air huarache women,"> air jordan 6, golden goose superstar,"> supreme shorts, yeezy boost 350, golden goose woman,"> air jordan 10,"> air jordan fusion,"> nike free run women, golden goose may, valentino pumps, golden goose mid star, adidas yeezy, golden goose ball star, balenciaga caps, valentino, golden goose may, moncler kids vests, valentino sneakers,"> nike air max 95 men, golden goose slide, golden goose v star, golden goose ball star,"> adidas harden vol 1,"> nike flyknit racer, alexander mcqueen, valentino sneakers, supreme clothing, supreme sweatshirts, balenciaga backpacks,"> nike lunar women,"> nike flyknit air max men, sport shoes

September 5, 2018 6:55 AM

chenlixiang said:



September 18, 2018 10:22 PM

happy wheelsgame said: happy wheels geometry dash

The rain brings sadness. And I'm sad that my web rankings are not as good as I expected. I need money for her to come to me! Feeling sad

September 22, 2018 1:15 AM

loemrntdherid1 said:

September 27, 2018 1:29 AM

chenjinyan said:



October 9, 2018 6:49 PM

chenqiuying said:


October 10, 2018 6:52 PM

chenlixiang said:


November 20, 2018 10:53 PM

eddie007 said:

There are the very nice post to all windows users have to look the online mozilla firefox free download latest version for the use full to all users

December 9, 2018 7:43 PM

Essaysrescue said:

Check out our list of essay writing services. And choose a company that is more suitable for your requirements. Absolute free review of the best companies in the world on writing

January 23, 2019 11:31 PM

Agen Poker Online Idn Play said:

This article is very good to read, a lot of knowledge can be compiled to read it

January 25, 2019 5:26 AM

Claudia Soo said:

Thank's For Your Information, Visit My Blog If you want to play Games Pokers. You Can Play And Win Everyday

<a href="">Daftar Poker Indonesia</a>

<a href="">Daftar Agen Poker IDNPLAY</a>

<a href="">Daftar Agen Poker Superten</a>

<a href="">Daftar Poker Omaha</a>

<a href="">Daftar Poker Online Terpercaya</a>

<a href="">Agen poker Idnplay resmi</a>

<a href="">Daftar Poker Online</a>

<a href="">Daftar Poker</a>

<a href="">Daftar Agen Poker</a>

<a href="">Daftar Situs Poker Online</a>

<a href="">Daftar Poker Idn</a>

<a href="">Daftar Agen Idnplay</a>

<a href="">Daftar Poker Resmi</a>

<a href="">Daftar Poker Online Resmi</a>

<a href="">Daftar Poker Idnplay</a>

<a href="">Daftar Poker Resmi IDNPLAY</a>

<a href="">Daftar Poker IDNPLAY Indonesia</a>

<a href="">Poker Online Idnplay</a>

<a href="">Game Poker Online Indonesia</a>

<a href="">Daftar Poker Indonesia</a>

<a href="">Daftar Poker IDN Play</a>

<a href="">Daftar Agen Poker Indonesia</a>

<a href="">Daftar Situs Poker Resmi</a>

<a href="">Daftar Situs Agen Poker</a>

<a href="">Daftar Game Poker IDNPLAY</a>

<a href="">Situs Poker Online Terpercaya</a>

<a href="">Situs Poker Online Indonesia</a>

<a href="">Situs Poker Idnplay</a>

<a href="">Situs Daftar Poker Idnplay</a>

<a href="">Situs Poker Online Idnplay</a>

<a href="">Situs Resmi Daftar Poker</a>

<a href="">Situs Agen Poker IDNPLAY</a>

January 25, 2019 8:52 AM

Claudia Soo said:

Thank's For Your Information, Visit My Blog If you want to play Games Pokers. You Can Play And Win Everyday, So Visit My Blog Now and Join Now.

January 25, 2019 8:54 AM

mary said:

Hi, it's really interested [url=][/url]

January 29, 2019 1:20 AM

arianecindy said:

Yeah! What a great blog you have published. I will share for my friends and tell him this is a good info. I like it and i will share it to others.

Visit my blog if u wanna play poker online at

February 1, 2019 6:48 PM

tiaclaudia said:

You have a very inspiring way of exploring and sharing thoughts. If yoou wanna play poker, come visit to my blog

February 2, 2019 1:16 PM

poker online idnplay said:

Very good information to share. Very useful to help with my work. Please visit to get easy money every day.

February 5, 2019 3:37 AM

Romi said:

Good blog indeed

February 7, 2019 1:27 AM

Bandar Judi Bola said:

Come and join us to betting online!">

February 9, 2019 11:46 PM

giselna said:

Nonton bola merupakan hal yang menyenangkan, apalagi jika ada taruhan uang asli di dalamnya. Maka dari itu, segeralah <a href="">daftar judi bola</a> di <a href="">bandar bola online resmi</a> untuk mulai meraup keuntungan melimpah. Di sana kalian dapat melakukan <a href="">taruhan mix parlay</a> dengan minimal bet mulai dari 10ribu rupiah dan minimal <a href="">deposit judi bola 50 ribu</a>. Untuk membantu Anda, kami memberikan <a href="">cara menang judi bola</a> di <a href="">situs agen bola terpercaya</a>. Sebagai <a href="">bandar judi bola terpercaya</a> 2019, kami juga menyediakan <a href="">bonus judi bola terbesar</a>.

February 10, 2019 12:02 AM

micalegria said:

yuk main capsa susun di !

daftar skarang juga

February 10, 2019 12:15 AM

sihotangjesika said:

come and join with us to playing poker online

February 10, 2019 12:22 AM

pamela said:

I really like this one thing. Very useful for increasing income, especially for families with middle to lower economies. I want to invite you to click if u want get some money.

February 16, 2019 1:48 PM

شركة نقل عفش بالطائف said:

February 18, 2019 4:15 PM

شركة نقل عفش بمكة said: نقل عفش داخل مكة نقل عفش داخل جدة نقل عفش داخل الباحة نقل عفش داخل الطائف نقل عفش داخل ينبع نقل عفش داخل نجران نقل عفش داخل خميس نقل عفش داخل ابها نقل عفش داخل الرياض نقل عفش داخل الاحساء نقل عفش داخل الدمام نقل عفش داخل تبوك نقل عفش داخل المدينة شركة نظافة داخل جدة نقل عفش داخل جازان نقل عفش داخل رابغ نقل عفش داخل الليث القنفدة شركات نقل عفش بالمنطقة العربية نقل عفش داخل الخبر نقل عفش داخل الحفر نقل عفش داخل الجبيل نقل عفش داخل حائل نقل عفش داخل بريدة نقل عفش داخل الخرج نقل عفش داخل القصيم نقل عفش داخل الجوف نقل عفش بالمنطقة الوسطى نقل عفش بالمنطقة الشرقية مكافحة حشرات بجدة نقل عفش بالمنطقة الجنوبية نقل عفش بالمنطقة الشمالية

February 18, 2019 4:23 PM

شركة نقل عفش بجدة said: شركة نقل عفش بالمدينة شركة نقل عفش بمكة شركة نقل عفش بالقصيم شركة نقل عفش بنجران شركة نقل عفش بالطائف شركة نقل عفش بالاحساء شركة نقل عفش بالجبيل شركة نقل عفش بتبوك شركة نقل عفش بالرياض شركة نقل عفش بجدة شركة نقل عفش بالدمام شركة نقل عفش بالخبر شركة نقل عفش بالباحة شركة نقل عفش بالحفر شركة نقل عفش بحائل شركة نقل عفش بينبع شركة نقل عفش برابغ شركة نقل عفش بجيزان شركات تنظيف في تبوك

February 18, 2019 4:24 PM

remote said:

Even if a database or other application had been architected to achieve a high degree of memory locality on a NUMA system, there is still a memory latency penalty on the multi-processor system relative to the single processor system.

Almost zero real-world databases have been architected to achieve memory locality on a NUMA system. I have heard that a handful of Oracle environments


Tas Seminar


Grosir Tas

jakarta indonesia

went to effort to re-architect for RAC, which is the same type of architecture necessary to achieve memory locality on NUMA. Most environments should see average memory latency

February 20, 2019 8:07 PM

Stefannie said:

Now is born a situs agen bola terbaik in Indonesia

March 2, 2019 2:49 AM

Jesica said:

Get your more Income at

March 2, 2019 2:51 AM

Venisia said:

the best agen poker idnplay in Indonesia

March 2, 2019 2:52 AM

honey said:

I love all the posts, I really enjoyed, I would like more information about this, because it is very nice., Thanks for sharing.

March 3, 2019 8:00 PM

bandar judi bola online said:

Hanya bermain situs judi bola terpercaya kamu bisa banyak meraih keuntungan di yuk mari bergabung bersama saya di

March 3, 2019 8:14 PM

situs judi bola terbesar said:

Terima kasih sekali lagi saya ucapkan atas waktu anda sekalian yang menyempatkan berkunjung di serta di

March 3, 2019 8:16 PM

saffa said:

<a href="">مركز صيانة زانوسى</a>

<a href="">مركز صيانة يونيون اير</a>

<a href="">مركز صيانة وايت بوينت</a>

<a href="">مركز صيانة هيتاشي</a>

<a href="">مركز صيانة سامسونج</a>

<a href="">مركز صيانة بوش</a>

<a href="">مركز صيانة كلفينيتور</a>

<a href="">مركز صيانة زانوسى</a>

<a href="">مركز صيانة الاسكا</a>

<a href="">مركز صيانة كاريير</a>

<a href="">مركز صيانة كريازي</a>

<a href="">مركز صيانة فريش</a>

<a href="">مركز صيانة جنرال اليكتريك</a>

<a href="">مركز صيانة اندست</a>

<a href="">مركز صيانة ال جى</a>

<a href="">مركز صيانة فريجيدير</a>

<a href="">مركز صيانة وستنجهاوس</a>

<a href="">مركز صيانة ويرلبول</a>

<a href="">مركز صيانة سيمنس</a>

<a href="">مركز صيانة هوفر</a>

<a href="">مركز صيانة دايو</a>

<a href="">مركز صيانة باناسونيك</a>

<a href="">مركز صيانة كريازي</a>

<a href="">مركز صيانة يونيون اير</a>

<a href="">مركز صيانة يونيفرسال</a>

<a href="">مركز صيانة ويرلبول</a>

<a href="">مركز صيانة وايت ويل</a>

<a href="">مركز صيانة وايت بوينت</a>

<a href="">مركز صيانة وستنجهاوس</a>

<a href="">مركز صيانة زانوسى</a

March 7, 2019 2:01 AM

daftar sbobet terpercaya said:

The quality of the content is great. See my blog to

March 8, 2019 7:39 AM

Bandar Togel Terpercaya said:

March 11, 2019 1:21 AM

anamika said:

It is one of the best articles that I have heard about have you seen latest updated post My topology is that Have you seen the latest Thank you so much for this latest updated blog.

March 12, 2019 2:16 AM

PAYAL said:

Modern Time May Job karma Kaliyah is attached to you more skill which meets Bangalore Insanity May, which is more than 10000 / May, which is a prince of Monthly, which can get a lot of work time.

Plz you can  visit to  Mg.Road , Bangalore and the  direct call to me : 7022789089

<a href="">bangalore escorts</a>

<a href="">escorts in Bangalore</a>

<a href="">bangalore escorts service</a>

<a href="">call girls in bangalore</a>

<a href="">bangalore escorts agency</a>

March 15, 2019 10:29 PM

PAYAL said:

The new time is the job of its services, which may be found in Bangalore, for which the Bangalore-based agency, everybody can get a piña pay work an kalian, the prize is very much work and the price is much higher.

Plz you can  visit to  Mg.Road , Bangalore and the  direct call to me : 7022789089

<a href="">bangalore escorts</a>

<a href="">escorts in Bangalore</a>

<a href="">bangalore escorts service</a>

<a href="">bangalore escorts agency</a>

<a href="">escort service in bangalore</a>

<a href="">bangalore escort girls</a>

<a href="">escorts service bangalore</a>

<a href="">escort service bangalore</a>

March 15, 2019 10:31 PM

vardir said:

I want to say thanks to you for allow us comment here

March 21, 2019 1:02 PM

Stuart Robins said:

The Above post is really impressive, as it shows why to choose iOS development platform in comparison to Android and any other one. It shows the best of iOS developement. But as per user perspective it is much needed to show how  to boost business with iOS app development and let your business grow on internet. For more info, reach at:

March 22, 2019 12:40 AM

Mark Gibb said:

Nice and informative post. You have really done marvelous job by sharing such a useful post.

March 22, 2019 10:33 PM

lukaka said:

I have followed your recipe great, delicious and quite easy to do compared to the previous recipe I did. Thank you very much for your cake

March 25, 2019 7:36 PM

Leave a Comment


About jchang

Reverse engineering the SQL Server Cost Based Optimizer (Query Optimizer), NUMA System Architecture, performance tools developer - SQL ExecStats, mucking with the data distribution statistics histogram - decoding STATS_STREAM, Parallel Execution plans, microprocessors, SSD, HDD, SAN, storage performance, performance modeling and prediction, database architecture, SQL Server engine

This Blog


Privacy Statement