Notes on community building

Taking part in a community building, the most important question is to decide if you are taking part building a true open Open Community or rather a Digital Apartheid. 

#Hashgraph

Notes on semi public blockchain networks, Hashgraph and regulation

There is an interesting combination between public and private blockchain networks and that is the sol called semi-public network. In such a field, the network will be operated or governed by several semi-trusted actors, but the network itself from a development or application point of view is open for everybody. However, it is pretty much questionable if that model can work at all. The model is that an application or transaction can only be censored if a majority of the consortium members will vote for that. It is good question what is going to happen on the practical side if an application is not conform with a regulation of a certain region. One model is that an application can only be filtered if most of the regions vote the application out. Exactly the opposite can happen however from the practical side as well, there might be the situation where an application has to be conform not just with one regulation of a certain region but with all regulation of all of the regions.   

How fair is the transaction ordering in Hashgraph

Hashgraph algorithm and Hedera Hashgraph is said to have a fair ordering mechanism that means that the order of the transactions is created with the help of a general consensus, so practically no node can influence the order of the transactions. It is an improved version of a blockchain mining protocol where a miner can very strongly influence the order of the transactions. 

The situation is actually not so cool. The point is that the algorithm guarantees a fair ordering of the events and not directly the transactions. As soon as a transaction is put into an event the fair ordering can be guaranteed however previously the ordering can be gamed. A node that receives transactions from the outside world can choose to include, not include or delay a transaction. It can also game with the ordering of the different transactions by delaying one with a couple of event round comparing with another one. So actually a node that receives the transaction is pretty much a dictator of that transaction, similarly as a miner is a dictator of the transactions put into a block. For such a "game" in the ordering of the transactions the original source code not necessarily should be modified. It is possible to hack the transactions on the operation system or on the network level.

Considering a Byzantine fault tolerant model meaning that at most one third of the nodes can be hacked, fair ordering of transactions can be only guaranteed if the transaction from the customer side is sent to 33% + 1 nodes.   

Hashgraph censorship

The Hashgraph technology is aimed to be a fair censorship resistant, byzantine fault tolerant technology having a public network as well, where theoretically there is no need to ask a permission form the company Swilrds or from consortium members of Hedera and the network will eventually as public as for instance Ethereum. Recent achievements  suggest however that the situation wont't be so clean and simple. The recent direction is to censor social media appearance that do not match into certain corporate policies, certainly the official names are "quality improvement" and "policy compliance", but actually most dictatorial governments have the same names censoring the local social media for filtering out the content that they do not like. Such a censorship raises some serious ethical questions even if it is carried out by a corporation, but it is surely not acceptable at an open distributed ledger platform. Censorship of the social media will very fast result in the censoring of individual transactions and decentralized applications on the platform. 

So, let we see how Hashgraph can censor your transactions:

- The easiest way is blocking incoming transaction at peer level, simply not including into the Hashgraph by dropping the incoming transaction. For that actually it is not necessary to modify the Hedera software itself, incoming transactions can be filtered out on the operation system or on the network level, like filtering out certain transactions from a certain IP address. Certainly, a customer will eventually recognize if a transaction is dropped and will resend, possibly eventual to another node. As a consequence censoring at peer level works theoretically only if all of the peers drop the transaction. However, from a practical point of view a customer will probably give up including the transaction in the Hashgraph after a certain number of attempts. On the other hand, resending too many, like 20, times is probably not acceptable for most of the applications and use cases, so from a practical point of view blocking transactions at several peers, like in a geographical area can make certain applications not-usable, even if from a theoretical point of the transaction is 100% blocked.

-   A similar censorship attack is to introducing a random delay for incoming transaction at the peer level. It is again something that can be carried out without modifying the official source code. If a transaction is 100% blocked, the customer will recognize it and resend it, possibly for another node. However if the transaction only delayed randomly it is more difficult to recognize. As a couple of minutes or hours random delay in the transaction processing is not acceptable for most of the decentralized applications, this can be an efficient denial of service attack for most of the distributed apps even if it is realized by one or two nodes.    

- As the transaction is included in an event, it is propagated with a gossip protocol throughout the network. You can use similar techniques here as well, like blocking event propagation that contains a certain transaction, or delaying and random blocking events containing a transaction. As the peers are connected with TSL, such an attack is difficult to realize by an external attacker, however it can be easily realized by the operators of the nodes as a censorship attack, again without modifying the code itself, implementing everything on the operation system or network side. Blocking a transaction or event will be efficient if more than one third will actively block, however at random delaying a much smaller number of number of censors could effectively make a DApp unusable. 

- There might be some further possible censorship possibilities at the state level and at the point as a transaction is applied to the state. As the implementation details are not known here, such attack vectors need further investigation.

- Last but not least, Hedera is basically a consortium that controls the code. It is pretty much an open question how the software upgrade mechanism will work what is sure that if the consortium votes against your transactions or DApp, a new software version can be delivered that will efficiently block your transactions and DApps. In practice, this process might be accelerated, as upgrading a new software version can be a slow process. I would probably create a configurable element in the software code itself where operators can individually set the blacklisted applications. It probably provides a further censorship attack vector.   

On chain governance with Hashgraph

Creating a native on-chain governance on Hashgraph is similar to creating any kind of an on-chain governance system.  For the first run there must be some special internal governance variables that might influence consensus parameters, node configuration or any other details of the core algorithm. Such variables should have two properties:

1. They can not be changed absolute freely only after a kind of a community voting. 

2. They must be anchored somehow with the nodes or with the core algorithm. 

For modifying on-chain properties there must be a special transaction, instead of a change value of a blockchain field transaction, there should be:

TR: I vote for changing the value of a governance variable - signed by the private key of an account that can actually vote for such a change. Such accounts might be associated with nodes, or with consortium members of the Hedera Hashgraph. 

After the voting transactions are validated and ordered and the voting period might be at the end, each node can apply the result to the state individually. A possible algorithm might be:

- count the votes for a given change, if it is more than 50% apply to the change to the state. 

  

Creating a Byzantine fault tolerant decentralized exchange with Hashgraph

The ordering algorithm of the Hashgraph architecture provides a way to realize a real decentralized exchange among different internal cryptoassets. Let we imagine that we have several native cryptoassets in a account/state based system. What we have to do is to introduce special transactions that represent buy and sell orders. As an example a buy order might look like:

Tr_order: buy 100 cryptoassets at 10 - signed by a private key of an account that has the balance of buying.

Similarly, a sell order might look like:

Tr_order: sell 50 cryptoassets at 11 - signed by the private key of an account that gets the cryptoasset at the end. 

The order transactions are validated and ordered with the help of the gossip of the gossip protocol and after a couple of rounds every node will see exactly the same order of the order transactions. This list of order transactions is actually the order book of the decentralized exchange. 

At applying the transactions to the state, we have to do two things: 

- there must be a kind of order matching algorithms on the order book, with other words on the ordered order transactions. Examples might be like FIFO or Pure Pro Rata. A list of possible order matching algorithms can be found for example under the following link.

- the orders that are matched has to be applied to the state. It practically means increasing and decreasing certain balances, similarly a simple transaction is applied to the state. In this case however there might be more than two accounts that has to be modified in the same time.

As the above mentioned algorithm is deterministic and the nodes see exactly the same order of order transaction, the nodes can calculate the algorithm independently from each other and getting the same result. As the core algorithm is Byzantine fault tolerant, the exchange will probably have the same property.

The decentralized exchange can be further improved in a way that instead of hard-coding the order matching algorithm, realizing an on-chain governance process to change order-matching on the fly. 

For experimental implementation take a look the following github repository.

Creating native decentralized oracle with Hashgraph

Hashgraph provides the possibility to realize a fully decentralized Byzantine native oracle service. Let we imagine that we want to call an external data source in a smart contract function, like

function () {
 state = external_oracle("http://external_datasource/%22)
}

At event propagation each {N1, N2, ... Nm} node has to call the http://external_datasource/ independently from each other producing a {V1, V2 ... Vm} possibly different values as results for the calls. These values are propagated, voted and ordered by the events together with the transactions. 

As soon as both the transactions and the values are communicated ordered and voted (so practically at applying to the state), we can use the following function to calculate the result of oracle: 

external_oracle calculation:
step1: filter from {V1, V2, .... Vm} the extreme big or small values, like 33% of all of the values
step2: do an average of the non-filtered values
step3: the result of the function is the average

Since at the time of the calculation each node sees exactly the same set of {V2, V2, ...Vm} values and the external_oracler calculation is deterministic, at each state there will be exactly the same value calculated even if they are calculated independently from each other. 

Since the communication architecture (Hashgraph) is Byzantine fault tolerant, probably the Oracle itself will be probably Byzantine fault tolerant as well (might require some more considerations)

The algorithm can be cryptoeconomically further secured, like giving a bigger transaction fee for nodes producing values that are less far for the end result of the external_oracle calculation function (providing a Schelling point from a game theoretical point of view).

Demo implementation can found in the following github repository.

Hashgraph, transaction censoring and transaction delaying

The Hashgraph algorithm provides from a mathematical point of view fair ordering and access. However it is important to point out that these mathematical proofs working on an event level, that might be a little bit different if we consider things on a transaction level or from a real architecure point of view.  

One attack might be to censor a newly created transaction on a node level, before the transaction even really reaches the network, before it will be put into a node. The censorship can be realized by directly hacking the Hedera software, but it might be realized indirectly as well, like censoring the transactions at the operation system or at the network level.  For such a censorship attack, there might be several resolution:

- users might use only trusted nodes. However in an untrusted or semitrusted architecture this is not a lucky algorithm. 

- users might broadcast a transaction to a node and monitor the system. If the transaction was not propagated, the transaction can be broadcasted again, or to another node, or several other nodes as well. It can work in situations where delaying or timing of the transactions can not cause problems. 

- to be 100% sure that the transaction is propagated as fast as possible, users has to broadcast the transaction to 33% +1 nodes. As the theoretical limit of a Byzantine fault tolerant architecture 33% in such a situation at least one node should work honestly and propagate the transaction right away.

Similar attack can be realized as a newly created transaction is propagated to the network by a node but not with the next possible event, but with a later one, cause indirectly a delay in the transaction. It might cause serious economical damage with certain timing relevant applications, like online gambling and games. To be sure that no delaying can occur with a newly created transaction, the most secure way is again to propagate the transaction to 33%+1 nodes in the system. In such situation, if one honest node propagates the transaction to the network as soon as it gets, than the transaction will be part of the hashgraph right away, independently if other faulty or hacked nodes delegate the transaction only with a delay.   

Fair ordering of transactions in Hashgraph

The Hashgraph virtual voting algorithm provides a fair ordering of the events in the following sense:

- Fair access: means that no individual can stop a transaction from entering the system,

or even delay it very much.

- Fair timestamps: A consensus timestamp that reflects when the majority of the network

members received that event. This consensus timestamp is “fair”, because it is not possible for a

malicious node to corrupt it and make it differ by very much from that time.

- Fair transaction order: A common order of the events that is to be seen exactly the same way after virtual voting by each node and that can not be influenced by individual nodes, or the minority of the nodes. 

There is one problem however, the mathematical side of the algorithms and the proofs guarantee a fair ordering of the events and not the transactions them self. Certainly, if each event contains at most one transaction, than fair ordering of the events equals with the fair ordering of the transactions. If however on event can contain multiply transactions, the situation is not so simple. A node creating multiply transaction for an event can game the order of the transactions in a certain event and even between multiply events. 

To avoid unfair ordering of transactions in an event an improvement might be to define an order and a timestamp of the transactions inside a node. On such an order might be to consider the hashes of the transactions in an event in a increasing order, like:

sha256(Tr_1), sha256(Tr_2), ... sha256(Tr_k) in an increasing order, where {Tr_1, Tr_2, ... Tr_k} is the set of transactions associated to an event. 

It does not really matter how an individual node orders the transactions in an event physically, the events are always regarded as the ordering of the transaction hashes. As a consequence it can not be gamed by the individual nodes unless transactions or signatures for transactions are forged that is usually pretty difficult because of the cryptographical algorithms.

Similarly a global order of the transactions can be defined as well:

- the global order of the events is provided by the Hashgraph algorithms.

- the order of the transactions in an event is provides by the ordering of the hashes. 

Based on the global order and the Hashgraph algorithm for attaching a timestamp for each event, the transactions can be associated by a timestamp as well:

- if an event contains only one transaction, the transaction timestamp equals with the timestamp of event. 

- if an event contains several transactions, the transactions must be timestamped in the event. A possible algorithm might be the following:

Consider an Ei event that contains the {Tr_1,Tr_2, .... Tr_j} transactions that are ordered as {sha256(Tr_1), sha256(Tr_2) ... sha256(Tr_j)}. The time period between Ei and Ei-1 can be evenly distributed among the transactions in a way that if a transaction gets a higher ordering, it gets a newer timestamp. 

Last but not least, a fair access can be only be reached if transaction are communicated to several nodes, like one 33% +1 of the exiting nodes.  

  

How to create real decentralized random oracle with Hashgraph

Real random oracle is one of unsolved Holy Grail of decentralized networks. Hashgraph technology promises perhaps a real possibility to solve it. A pseudo algorithm should be the following:

Let we assume that we have a smart contract function that assigns to a state a new value, like

function() {
 state = new_state
}

we basically do not call this function but create a transaction, like

TR: call function() signed by private_key of some account owner.

This transaction is propagated on the network by the gossip of gossip protocol, probably checked by every node, goes through a virtual voting and ordered with other transactions into a common order. Only after that, it is applied to the state (independently if it is an account based system or an UTXO based).

so let assume a function that calls a real native decentralized random oracle, like the following function:

function() {
 state = rand()
}

and a transaction that calls this function.

As soon as a node "sees" the transaction of function that contains a random oracle call, the node call its internal native random function and creates a ri random value, which is further propagated on the network similarly as the transactions. As a consequence there will be a set of {r1, r2, .... rN} random values propagated on the network, one for each node.

As a next step, we wait until there is a consensus and an order of these random values, so every node will see exactly the same order of these individual random values, like r4, r3, r1, r2 ... rN. And at this point we give a value to the random function as rand() = merkle_root(r4, r3, r1, r2 ... rN). As merkle_root is a deterministic function and and each node sees exactly the same order of individual random numbers, even if rand() is calculated independently at each node it will produce the same result.  

Certainly, the real quality depends on the individual random numbers as well. However as Hashgraph is Byzantine fault tolerant, if more than two third of the nodes provide a relative good random number meaning either cryptographically secure or pseudorandom, the result will have probably the same attribute, meaning cryptographically secure or  pseudorandom.

Experimental implementation can be found in the following github repo.

Types of blockchain governance

One of the critical part of every blockchain solution is governance. It is especially valid for blockchain platforms, like Bitcoin or Ethereum, but it is actually the same for blockchain applications that are realized on the top of a platform. Governance means carrying out changes or further development of the application. There are several models that has different advantages and disadvantages in the practice:

- Off-chain governance means that the governance model is totally independent from the blockchain itself. Changes of the platform is proposed by a community or trusted people and both the implementation and the implementation is carried out by these people as well. Such a model has the advantage that it is technologically less challenging, however it might cause a split in the community resulting a split or hard-fork in the platform itself as well. Such models are realized by Bitcoin and Ethereum where improvements are presented by improvement proposals, like Bitcoin Improvement Proposal (BIP), or Ethereum Improvement Proposal (EIP). The model caused several hard-forks in the above mentioned platforms, resulting for example Bitcoin Cash, or Ethereum Classic. 

- On-chain governance means that several elements of the platform is able to change by a community decision, and both the decision mechanism and the change implementation is supported by the platform itself. Such an example is the difficulty increase or decrease mechanism of the Bitcoin network which is hard-coded in the consensus mechanism and is practically relized by the voting of the miners or the voting of the stakeholders. Other platforms like Monero have much further developed on-chain consensus mechanism, as an example part of the transaction fee is collected into an online wallet as an improvement budget, which can be used for further development or even marketing projects based on the voting of the community. 

From another perspective blockchain governance can be categorized on the fact, who makes the decision. From this point of view there might be the following models: 

- Centralized governance: systems like Ethereum has a pretty centralized governance. From a practical point of view almost all of the changes are governed eventually by Buterin. It has certainly, the advantage of being able to adapt fast, however not necessarily truly open as participation. 

- Open governance: Systems like Bitcoin has a real open governance model. Practically everybody can initiate new changes in the system and the implementation of these changes require a pretty open consensus as well. Practically the core developers, the miners, the exchanges and the shops should all agree in order that a change is implemented successfully. As it is almost a pretty open democratic process, certainly it can be very slow and inefficient in the implementation phase.

- Governance by trusted actors: There can be model between the two previously mentioned examples as well, where several trusted (and sometimes well known) actors can initiate and implement changes in the platform. Such governance model will be implemented directly by Hedera Hashgraph. Similar initiative can be found like in Dash or Monero, where indirectly the stakeholders or the master node responsibles can be regarded as trusted actors.    

Hashgraph SDK Tips and Tricks - implementing your first application

If you already started to implement your first application with the Hedera Hashgraph SDK as it was summarized in the previous blog, you should continue with the following:

1. Extend your State class with additional elements storing state variables, like lists, arrays, hashmaps.

2. Rewrite the copy functions of your State class to cover the newly introduced state variables. 

3. Implement custom query functions in the State class to query the newly introduced state variables. 

4. Rewrite the init function of the Main class, to use any kind of custom initialization. Like, you can query the setup up parameters of the config file with platform.getParameters(), or you can create new consoles or windows with the platform.createConsole(true) and platform.createWindow(true) functions.

5. Rewrite the run() function of the Main class to run any kind of the required custom logic for your node. 

Hashgraph SDK Tips and Tricks - starting with your first application

If you start with your first application with Hedera Hashgraph, the easiest way is to download the SDK from the homepage and copy and start modifying one of the existing applications. For this:

1. Download the project from : https://www.swirlds.com/download/

2. Copy one of the applications under  sdk/source

3. Rename the folder and rename the name references in the pom.xml file.

4. Open Eclipse

5. Import the files as a "Maven" project

6. Rename the Main and State java files, objects and all the references to the name of your projects

7. Press "Run As" as "Maven Install" to get the binaries. 

8. Configure your application in the config.txt file. 

9. Start and test your application with swirlds.

Hashgraph SDK Tips and Tricks - logging error

If you use the Hashgraph SDK demo applications for prototype development, you can recognize that debugging the applications is not so simple. What you can do is to start a separate console window, just for the errors and you can log every important ´piece of information or errors to this special console, like:

/** a console window for errors */
public Console errorconsole;
...
  public void init(Platform platform, long id) {
  ...
    this.errorconsole = platform.createConsole(true);
  ...
  }

  protected void LogException(Exception e) {
    errorconsole.out.println(e.toString()+" "+e.getMessage()+" ");
  }

  protected void LogMessage(String message) {
    errorconsole.out.println(message);
  }

Unfortunately, it does not work if you want to log from your state object. 

Notes on decentralized artificial intelligence

Current trends in artificial intelligence, like Singularity.Net, Neureal or Pandora having a little bit the wrong direction. Overall is the idea, that different artificial intelligence components can be somehow marketed and sold with the help of a marketplace which is realized with the help of a blockchain solution and some tokens. However this idea is not really optimal. Distributed ledger should actively help the integration of the different artificial intelligence component, effectively realizing a hierarchical pattern recognition structure. Tokens should not only represent an economic incentive but they should actively carry information among the different pattern recognition components to effectively function as a data bus of the architecture. Certainly classical architectures like blockchain can not really scalled up to carry such a huge amount of information, however with technologies like Hashgraph functioning solution might be carried out.

Certainly, it is a major question if distributed ledger technology is really required, or possibly there is the chance to put everything to a centralized data bus. 

On fairness of transaction ordering in blockchains

Proof of work and mining systems are actually pretty far from being fair regarding transaction ordering. On the one hand the miners in validators in a Proof of Stake system work as local dictators on the set of transactions, meaning that they can select which transactions are put into the next block. This gives on the one hand the possibility to censor or delay certain transactions. How big this delay might be depends on the competition and collaboration of the miners and validators. On the other hand, as the transactions are usually processed based on transaction fees and miners certainly priories the transactions with the higher transaction fee. This gives the possibility for an average user to game the system, in the sense that it can be sure that a higher fee transaction will be surely prioritized over a lower transaction fee, giving for instance the possibility for a successful double spending attack. 

It is an open question if a blockchain protocol can designed in a fair ordering way. Other distributed ledger technologies like Hashgraph has the fair ordering property.  

public - semi public and consortium blockchain platforms

In usual blockchain frameworks there are two major ways of defining the general visibility:

- public blockchain: the nodes can be run by anybody, even by hackers. The transactions are visible to everybody as well and practically the whole world can participate. 

- consortium blockchain: the nodes are operated by certain consortium members, and these members have access to the transactions and the whole blockchain content as well. 

There might be a third solution between public and consortium scenarios, let we call it a semi public blockchain. In this version the peers are operated by a consortium with known members, but the network itself fully public meaning that everybody can participate on the transaction side and everybody can deploy and operate smart contracts or decentralized applications on top of the network without requiring a permission from the consortium of the infrastructure side. Such scenario is proposed for example by the Hedera Hashgraph. 

Comparing Ethereum with Hashgraph

Based on the framework of the previous blog and dimensions, we can compare Ethereum with the new emerging technology called Hashgraph. The characteristics of Ethereum can be seen on the following picture:

And similarly the characteristics of Hashgraph can be seen on the following picture:

Comparing the two frameworks, we can see that there are differences in two areas. First of all Hashgraph is at the moment rather an extended consensus mechanism and not a full scale distributed ledger technology. That means that some elements are simply not defined exactly by the framework and are pretty much open for further implementation. Such things are related to the exact transaction realization.The computational part of the consensus can be pretty easily extended in java, however how exactly transactions are implemented is a point that provides the possibility to several implementations. In this sense properties like crypto-economics transaction semantics or transaction privacy are parameters that provide several possibilities in the future. The sure thing is that contrary to Ethereum Hashgraph best fit in consortium scenarios, the structure of the transaction storage is a non-blockchain one and both performance and fault tolerance are more than excellent.

Certainly, this article covers the original concept of Hashgraph produced by Swirlds, in the public version of the technology produced by the Hedera consortium it is planned to have more structure and services regarding on transaction, cryptocurrecnies and storage.

Proof of Useful Work in Hashgraph

Hashgraph algorithms actually do not really need a Proof of Work algorithm, the voting itself can be realized by the graph itself with a relative minimal cost, the only issue that remain how the system should avoid Sybil attacks. Here one of the idea might be to use some kind of a tokens for voting and the token distribution can be controlled in a way. One of the idea for the voting token distribution can be simply a consortium network scenario, the other one is a something similar to a Proof of Stake. 

However the voting token distribution can be controlled with the help of a Proof of useful Work algorithm as well. Everyone who did some amount of useful work get tokens in the system to vote. Such a mechanism provide several advantages comparing to classical Proof of Work of a Blockchain system. In Hashgraph Proof of Work we do not need rounds, so the work itself must be not scalable to blocks, it might vary and take largely different amount of time frames. On the other hand, the proof of work should not be competitive as in Bitcoin for example where the fastest wins and the rest of the efforts will be lost. Here, practically every piece of work will be regarded with tokens, so they are considered as valid.    

Certainly, there remain some open question. On the one hand, how can be proved that the amount of work was really done. On the other hand, it must be analysed by game theoretical perspective if maintaining the system is a Nash equilibrium. 

The end of Blockchain .... long live the Hashgraph ?

Without any Proof of "Waste" algorithm, without incentives, without transaction fee, proovable Byzantine, implicit decentralized voting with limited network  usage, maximally scallable ....

Actually, I still did not read the proofs but if only half is true what they claim, this is gonna be a big one.